Skip to content

pulsar_deployment_guide.md

Latest commit

 

History

History
88 lines (55 loc) · 3.18 KB

pulsar_deployment_guide.md

File metadata and controls

88 lines (55 loc) · 3.18 KB

Overview

A complete Pulsar cluster is composed of Pulsar broker, necessary ZooKeeper and BookKeeper services. The specific deployment methods can be divided into two types, they are:

  1. Standalone, all services are started on one host, easy to deploy, suitable for local development and testing, and can also be used in some production environments.

  2. Cluster, the service is scattered on different hosts, the deployment is slightly complicated, but it has the characteristics of high availability, load balancing, etc. It is suitable for the production environment.

1. Prerequisite

  1. Java 8
  2. Linux operating system

2. Download the Pulsar 2.7.0 installation package

Download from Pulsar official website download page

or

Use the wget command to download:

$ wget https://archive.apache.org/dist/pulsar/pulsar-2.7.0/apache-pulsar-2.7.0-bin.tar.gz

After downloading the compressed file, unzip it to "/data/projects/fate/common" and use the cd command to enter the target directory:

$ tar -xvfz apache-pulsar-2.7.0-bin.tar.gz -C /data/projects/fate/common
$ cd /data/projects/fate/common/apache-pulsar-2.7.0

3. Start the Standalone Pulsar Service

  1. Edit the "conf/standalone.conf" file and modify it as follows:
# Enable to delete inactive topic
brokerDeleteInactiveTopicsEnabled=true

# Setting time interval for scanning of inactive topic
brokerDeleteInactiveTopicsFrequencySeconds=36000

# Setting number of connection for replication
replicationConnectionsPerBroker=4

# Modify the frame size of bookkie to 128MB, the default size is 5MB
nettyMaxFrameSizeBytes=134217728

# Modify the size of the pulsar message capacity to 128MB, the default is 5MB
maxMessageSize=134217728

# Default per-topic backlog quota limit(-1 means no limit)
backlogQuotaDefaultLimitGB=-1

For more details about the configuration, please refer to the file mentioned above.

  1. Edit the "conf/pulsar_env.sh" file and modify it as follows：
# Extra options to be passed to the jvm
PULSAR_MEM=${PULSAR_MEM:-"-Xms4g -Xmx4g -XX:MaxDirectMemorySize=8g"}
  1. Start the pulsar service
$ bin/bin/pulsar-daemon start standalone -nss

The running log is in the logs directory under the pulsar software deployment directory .

  1. Stop the pulsar service

When the pulsar cluster is no longer needed, the following command can be used to stop it.

$ bin/pulsar-daemon stop standalone

When the data generated by the pulsar service is no longer used, the user can remove "apache-pulsar-2.7.0/data" directory.

For more information about the configuration of the standalone version of Pulsar, please refer to the official Pulsar document Set up a standalone Pulsar locally

4. Start the cluster version of Pulsar cluster

For Pulsar service deployment in cluster mode, please refer to the official document Deploying a multi-cluster on bare metal, Among them, Deployment Configuration store can be skipped.