<< Click here to display Table of Contents >>

Installing Index Server Cluster on Linux

Contents

NetBrain Index Server adopts Elasticsearch as a full-text search and analytics engine in a distributed multi-user environment. You need to know the following concepts before deploying an Index Server Cluster.

Cluster

A cluster is a collection of one or more nodes (servers) that together holds your entire data and provides federated indexing and search capabilities across all nodes.

A cluster is identified by a unique name which by default is "elasticsearch". This name is important because a node can only be part of a cluster if the node is set up to join the cluster by its name.

Note: Make sure that the cluster name is unique in the same LAN.

Shard

Each index in Elasticsearch is divided into shards and each shard can have multiple copies. These copies are known as a replication group. Each shard is in itself a fully-functional and independent "index" that can be hosted on any node in the cluster.

Sharding is important for two primary reasons:

▪It allows you to horizontally split/scale your content volume.

▪It allows you to distribute and parallelize operations across shards (potentially on multiple nodes) thus increasing performance/throughput.

When you create an index, you can simply define the number of shards that you want. By default, each index is allocated with 5 primary shards and 1 replica.

Node

By default, each node is a master-eligible node, which has the chance to be selected as the master node through the master election process. A master node takes the responsibilities for lightweight cluster-wide actions, such as creating or deleting an index, tracking which nodes are part of the cluster, and deciding which shards to allocate to which nodes.

In a cluster deployment, you can configure two node types:

▪Master Node — a master-eligible node and holds the shards that contain the documents you have indexed and handles related data operations, such as CRUD, and search.

▪Master-only Node — a master-eligible node with the hold data ability disabled.

Installing an Index Server Cluster

The following example introduces how to deploy a three-node cluster for high availability. Two of them are master nodes, and another one is a master-only node.

The master nodes back up the data of each other through shards. No data will be lost when one of the master nodes is unavailable. Even if both the master-only node and one of the master nodes are unavailable, the other master node can serve the search service as normal.

Note: Index Server Cluster requires the port TCP 9300 for internal node-to-node communication.

1.Complete the following steps to install each Index Server node.

Note: Don't use the machine on which Elasticsearch from other channels has been installed. Elasticsearch has a dependency on openjdk-11.0.1, which will be automatically installed while Elasticsearch is installed. If another version of OpenJDK or JDK has been installed on this machine, it will be automatically uninstalled before openjdk-11.0.1 is installed.

Note: Install master nodes first, and then install master-only. To successfully start the Elasticsearch service on either of the master nodes, you must run the ./install.sh script to install them synchronously.

Note: It is recommended to install the master-only node on the arbiter node of MongoDB replica set (in this example, 10.10.3.144). Besides, make sure that your network configurations allow communications among all nodes.

1)Log in to the Linux server where a MongoDB node is installed as the root user.

2)Obtain the elasticsearch-linux-rhel7-6.5.2.tar.gz file from NetBrain, and then copy it to the /opt/netbraintemp directory.

Note: Don't place the installation package under any personal directories, such as /root.

▪Option 1: If the Linux server has no access to the Internet, upload the elasticsearch-linux-rhel7-6.5.2.tar.gz file to the /opt/netbraintemp directory by using a file transfer tool.

▪Option 2: If the Linux server has access to the Internet and the yum command has been installed:

a)Run the cd /opt/netbraintemp command to navigate to the /opt/netbraintemp directory.

b)Run the wget <NetBrain IE 7.1a2 Elasticsearch download link> command under the /opt/netbraintemp directory to directly download the elasticsearch-linux-rhel7-6.5.2.tar.gz file to this directory.

Tip: Contact NetBrain Support Team to get the download link address.

Note: The download link is case-sensitive.

3)Run the tar -zxvf elasticsearch-linux-rhel7-6.5.2.tar.gz command under the /opt/netbraintemp directory to extract installation files.

[root@localhost netbraintemp]# tar -xvf elasticsearch-linux-rhel7-6.5.2.tar.gz
Elasticsearch/
Elasticsearch/cert.pem
Elasticsearch/elasticsearch-6.5.2.tar.gz
Elasticsearch/elasticsearchconfig.sh
Elasticsearch/fix_releaseinfo.json
Elasticsearch/install.sh
Elasticsearch/installelasticsearch.sh
Elasticsearch/install_elasticsearch.conf
Elasticsearch/jdk.sh
Elasticsearch/node.key
Elasticsearch/openjdk-11.0.1_linux-x64_bin.tar.gz
Elasticsearch/preinstallcomponents/
Elasticsearch/preinstallcomponents/CentOS7/
Elasticsearch/preinstallcomponents/CentOS7/lsof-4.87-4.el7.x86_64.rpm
Elasticsearch/preinstallcomponents/RedHat7/
Elasticsearch/preinstallcomponents/RedHat7/lsof-4.87-4.el7.x86_64.rpm
Elasticsearch/search-guard-6-6.5.2-23.2.zip
Elasticsearch/servicename
Elasticsearch/uninstallelasticsearch.sh
Elasticsearch/upgrade.sh
Elasticsearch/userpassword.sh

4)Run the cd Elasticsearch command to navigate to the Elasticsearch directory.

5)Modify the parameters in the install_elasticsearch.conf file located under the Elasticsearch directory according to your environment and save the changes. For how to modify the configuration file, see Appendix: Editing a File with VI Editor for more details.

#NetBrain elasticsearch config file

ESSystemUser                    netbrain
ESSystemGroup                   netbrain
InstallPath                     /home/elasticsearch
ServiceName                     elasticsearch
#Each member in the cluster must have the same ClusterName, Username, and Password.
ClusterName                     elastic-search-cluster
NodeName                        node1
#The Username or Password should not contain: {}[]:",'|<>@&^%\ or a space. The length cannot exceed 64 characters.
User                            admin
Password                        admin

DataPath                        /home/elasticsearch/data
LogPath                         /home/elasticsearch/logs
BindIp                          10.10.3.142
#The port must be between 1025 and 65535.
Port                            9200

#The maximum system resources that can be occupied by elasticsearch: CPUlimit(%), MemoryLimit(GB)
CPULimit                        35%
MemoryLimit                     4

UseSSL                          no
CertAndKeyPath                  /etc/ssl
PrivateKey                      node.key
Certificate                     cert.pem
CertAuth                        root-ca.pem
#single node or cluster
SingleNode                      no
#Write all cluster members.
ClusterMembers                  10.10.3.142 10.10.3.143 10.10.3.144
MasterOnlyNode                  no

6)Run the ./install.sh command under the Elasticsearch directory to install Elasticsearch and initialize the username as well as password.

[root@localhost Elasticsearch]# ./install.sh
Linux localhost.localdomain 3.10.0-327.el7.x86_64 #1 SMP Thu Nov 19 22:10:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
It may take a few minutes to install JDK. Please wait.
Successfully installed openjdk-11.0.1.
It may take a few minutes to install the elasticsearch. Please wait.
Successfully installed the elasticsearch.
Successfully modified the configurations of the elasticsearch.
Starting the service of the elasticsearch. Please wait...
Successfully initialized the username and password in the elasticsearch.
It is the No.1 times to attempt to connect to the elasticsearch, please wait...
Successfully connected to the elasticsearch. The setup was finished.

2.Complete the following steps to verify the cluster.

1)Enter http(s)://<IP address of any Elasticsearch node>:<port number>/_cat/nodes?v in the address bar of your web browser. For example, http://10.10.3.142:9200/_cat/nodes?v.

2)Enter the login username and password that you set up in the install_elasticsearch.conf file and click Login to authenticate your credentials. The default credentials are admin/admin.

3)If the login succeeds, you can browse the Index Server information on the login page. Here is an example:

The node.role column indicates the role that each node plays in the cluster.

om indicates the node is used as a master-only node.

omdi indicates the node is used as a master node.

oThe * character indicates the location of the current elected master. A cluster can only have one master at any time.

Index Server Parameters

Refer to the following table for the parameters of Index Server.

Parameter	Default Value	Description
ESSystemUser	netbrain	The system user created on the Linux server to run the service of the Index Server. Keep the default name as it is. Note: The values of the ESSystemUser and ESSystemGroup parameters can only contain letters and numbers, and must start with a letter. The same rule applies to the ServiceName, ClusterName, and NodeName parameters. Note: If you modified the default umask value, make sure this system user has the read and execute permissions to all the directories (including their parent directories) where the Index Server is installed. These directories include that you specify in InstallPath, DataPath, LogPath, and CertAndKeyPath.
ESSystemGroup	netbrain	The system group created on the Linux server to run the service of the Index Server. Keep the default name as it is.
InstallPath	/opt/elasticsearch	The storage path for all binary and other installation files of the Index Server. Keep the default path as it is. Note: If you modified the default umask value, you must run the chmod o+rx /opt/elasticsearch command to assign the read and execute permissions of the /opt/elasticsearch/ directory to the system user that you specify in ESSystemUser. This note also applies to DataPath, LogPath, and CertAndKeyPath. Note: Don't set the value of the InstallPath parameter to any personal directories, such as /root. Besides, don't include any special characters or spaces except slashes (/). The same rule applies to the DataPath, LogPath, and CertAndKeyPath parameters.
ServiceName	elasticsearch	The service name of the Index Server. Keep the default name as it is.
ClusterName	elastic-search-cluster	A unique name of the cluster. Keep the default name as it is. Note: Each member in a cluster must have the same cluster name, username, and password.
NodeName	node1	A unique name of the node in a cluster. Keep the default name as it is if you deploy a standalone Index Server.
User	admin	The admin username used to log in to the Index Server. Note: The username and password cannot contain any of the following special characters, and its length cannot exceed 64 characters. { } [ ] : " , ' \| < > @ & ^ % \ and spaces
Password	admin	The admin password used to log in to the Index Server.
DataPath	/opt/elasticsearch/data	The storage path for all data files of the Index Server. Note: It is recommended to keep the default path as it is. If you want to modify it, don't use an existing directory and make sure the directory has more than 50GB free space to save all the data files. Tip: You can run the df -h command to check which directory has been mounted to a large disk.
LogPath	/opt/elasticsearch/logs	The storage path for all log files of the Index Server. Note: It is recommended to keep the default path as it is. If you want to modify it, don't use an existing directory and make sure the directory has more than 10GB free space to save all the log files.
BindIp	192.168.1.1	The IP address of the Index Server. Note: If you modify the parameter to 0.0.0.0, you must use the fully qualified domain name (FQDN) to connect to the Index Server when installing API Server and Worker Server. Note: Don't use 0.0.0.0 if you want to deploy a cluster.
Port	9200	The port number that the service of the Index Server listens to. Keep the default value as it is.
CPULimit	35%	The maximum CPU utilization that can be consumed by the Index Server. To make both MongoDB Server and Index Server utilize the CPU resources of the same machine, the recommended value is 35%. And the sum of CPU utilization allocated to the MongoDB Server and Index Server cannot exceed 90% of the machine's CPU.
MemoryLimit	4	The maximum memory capacity of the machine that can be consumed by the Index Server. To make both the MongoDB Server and Index Server utilize the memory resources of the same machine, the memory allocated to the Index Server must be in the range of 1/8 to 1/4 of the physical memory of the machine. If the machine's memory is 16GB, the recommended value is 4; if the machine's memory is 32GB, you can modify the value to 8. Note: The maximum memory that Index Server can utilize is 31GB. Setting the value of the MemoryLimit parameter to higher than 31 will not increase the performance of Index Server. Instead, it may affect the performance of co-existing servers on this machine.
UseSSL	no	Set whether to enable the encrypted connections to Index Server by using SSL. For detailed requirements of SSL certificates and keys, see Appendix for more details.
CertAndKeyPath	/etc/ssl	The storage path for all the certificates and key files used for SSL authentication. Note: It is required only if UseSSL is enabled.
PrivateKey	node.key	The name of SSL private key file. Note: It is required only if UseSSL is enabled.
Certificate	cert.pem	The name of SSL certificate file containing the public key. Note: It is required only if UseSSL is enabled.
CertAuth	root-ca.pem	The name of SSL certificate chain or intermediate certificate (class 2 or class 3 certificate). Note: It is required only if UseSSL is enabled.
SingleNode	yes	Set whether to enable the cluster deployment. The default option yes means cluster deployment is disabled. Change it to no if you want to deploy a cluster. Keep the default option as it is if you deploy a standalone Index Server.
ClusterMembers	10.10.2.34 10.10.2.35 10.10.2.36	Note: This is required only for cluster deployments. Keep the default value as it is if you deploy a standalone Index Server. Enter the IP addresses of three cluster members if you want to deploy a cluster. Note: The first entered cluster member has the tendency to be elected as the master but the master-only node cannot be the first member.
MasterOnlyNode	no	Note: It is required only for cluster deployments. The default option no means this node will be used as a master node. If a master-only node is required, it is recommended to change the value to yes on the arbiter node of the MongoDB replica set. Keep the default option as it is if you deploy a standalone Index Server.