NetworkBrain Index Server adopts Elasticsearch as a full-text search and analytics engine in a distributed multi-user environment. You need to get to know the following concepts before deploying an Index Server Cluster.

Cluster

A cluster is a collection of one or more nodes (servers) that together holds your entire data and provides federated indexing and search capabilities across all nodes.

A cluster is identified by a unique name which by default is "elasticsearch". This name is important because a node can only be part of a cluster if the node is set up to join the cluster by its name.

Note: Make sure that the cluster name is unique in the same LAN, otherwise you might end up with nodes joining the wrong cluster.

Shard

Each index in Elasticsearch is divided into shards and each shard can have multiple copies. These copies are known as a replication group. Each shard is in itself a fully-functional and independent "index" that can be hosted on any node in the cluster.

Sharding is important for two primary reasons:

▪It allows you to horizontally split/scale your content volume.

▪It allows you to distribute and parallelize operations across shards (potentially on multiple nodes) thus increasing performance/throughput.

When you create an index, you can simply define the number of shards that you want. By default, each index is allocated with 5 primary shards and 1 replica.

Node

By default, each node is a master-eligible node, which has the chance to be selected as the master node through the master election process. A master node takes the responsibilities for lightweight cluster-wide actions, such as creating or deleting an index, tracking which nodes are part of the cluster, and deciding which shards to allocate to which nodes.

In a cluster deployment, you can configure two node types:

▪Master Node — a master-eligible node and holds the shards that contain the documents you have indexed and handles data related operations, like CRUD, search, and so on.

▪Master-only Node — a master-eligible node with the hold data ability disabled.

Installing an Index Server Cluster

The following example introduces how to deploy a three-node cluster for high availability. Two of them are master nodes, and another one is a master-only node.

The master nodes back up the data of each other through shards. No data will be lost when one of the master nodes is unavailable. Even if both the master-only node and one of the master nodes are unavailable, the other master node can serve the search service as normal.

Note: Index Server Cluster requires the port TCP 9300 for internal node-to-node communication.

1.Complete the following steps to install each Index Server node on the Linux server separately.

Note: Don't use the machine on which Elasticsearch from other channels has been installed.

Note: Install master nodes first, and install master-only in sequence. Wait until the installation is finished on one node and then start to install the other one.

Note: It is recommended to install the master-only node on the arbiter node of MongoDB replica set (in this example, 10.10.3.144). Besides, make sure that your network configurations allow communications among all nodes.

1)Log in to the Linux server as root user.

Note: Index Server has a dependency on jdk1.8.0_131, which will be automatically installed during Index Server installation. To check if another JDK version is installed on this machine, run the java -version command.

- If the returned result shows java version "1.8.0_131" or command not found, continue with step 2).

- If the returned result shows another JDK version, you can either uninstall all existing JDK versions, or install and configure jdk1.8.0_131 to make sure the java -version command returns 1.8.0_131. Then continue with step 2).

2)Run the cd /opt/netbraintemp command to navigate to the /opt/netbraintemp directory.

3)Copy the Elasticsearch.tar file into the /opt/netbraintemp directory.

Note: Don't place the installation package under any personal directories, such as /root.

▪Option 1: If the Linux server doesn't have access to the Internet:

Obtain the Elasticsearch.tar file in the installation package and upload it to the /opt/netbraintemp directory that you created by using a file transferring tool.

▪Option 2: If the Linux server has access to the Internet:

Run the cp Elasticsearch.tar /opt/netbraintemp command under the download directory in Section 2.1.2 to copy the Elasticsearch.tar file to the /opt/netbraintemp directory.

4)Extract installation files from the Elasticsearch.tar file by running the tar -xvf Elasticsearch.tar command under the /opt/netbraintemp directory.

[root@localhost netbraintemp]# tar -xvf Elasticsearch.tar
Elasticsearch/
Elasticsearch/preinstallcomponents/
Elasticsearch/preinstallcomponents/RedHat7/
Elasticsearch/preinstallcomponents/RedHat7/lsof-4.87-4.el7.x86_64.rpm
Elasticsearch/preinstallcomponents/CentOS7/
Elasticsearch/preinstallcomponents/CentOS7/lsof-4.87-4.el7.x86_64.rpm
Elasticsearch/elasticsearch-6.0.0.tar.gz
Elasticsearch/jdk-8u131-linux-x64.tar.gz
Elasticsearch/servicename
Elasticsearch/search-guard-6-6.0.0-17.beta1.zip
Elasticsearch/cert.pem
Elasticsearch/node.key
Elasticsearch/root-ca.pem
Elasticsearch/jdk.sh
Elasticsearch/elasticsearchconfig.sh
Elasticsearch/userpassword.sh
Elasticsearch/install_elasticsearch.conf
Elasticsearch/uninstallelasticsearch.sh
Elasticsearch/install.sh
Elasticsearch/installelasticsearch.sh

5)Run the cd Elasticsearch command to navigate to the Elasticsearch directory.

6)Modify the following parameters in the install_elasticsearch.conf file located under the Elasticsearch directory based on your needs and save the changes. See Appendix: Editing a File with VI Editor for more details.

#NetBrain elasticsearch config file
ESSystemUser                    elasticsearch
ESSystemGroup                   elasticsearch
InstallPath                     /opt/elasticsearch
ServiceName                     elasticsearch
ClusterName                     elastic-search-cluster
NodeName                        node1
#The Username or Password should not contain: {}[]:",'|<>@&^%\ or a space. The length cannot exceed 64 characters.
User                            admin
Password                        admin
DataPath                        /opt/elasticsearch/data
LogPath                         /opt/elasticsearch/logs
BindIp                          10.10.3.142
Port                            9200
#The maximum system resources that can be occupied by elasticsearch: CPUlimit(%), MemoryLimit(GB)
CPULimit                        35%
MemoryLimit                     4
UseSSL                          no
CertAndKeyPath                  /etc/ssl
PrivateKey                      node.key
Certificate                     cert.pem
CertAuth                        root-ca.pem
#single node or cluster
SingleNode                      no
#Write all cluster members.
ClusterMembers                  10.10.3.142 10.10.3.143 10.10.3.144
MasterOnlyNode                  no

Note: The ESSystemUser, ESSystemGroup, ServiceName, ClusterName, and NodeName arguments can only contain letters and numbers, and must start with a letter.

▪ESSystemUser — the system user created on the Linux server to run the service of the Index Server. Keep the default name as it is.

▪ESSystemGroup — the system user created on the Linux server to run the service of the Index Server. Keep the default name as it is.

▪InstallPath — the storage path for all binary and other installation files of the Index Server. Keep the default path as it is.

Note: Don't set the InstallPath argument under any personal directories, such as /root, or contain any special characters or spaces except slashes (/). The same rule applies to the DataPath, LogPath, and CertAndKeyPath arguments.

▪ServiceName — the service name of the Index Server. Keep the default name as it is.

▪ClusterName — a unique name of the cluster. Keep the default name as it is.

Note: Each member in the cluster must have the same cluster name, username, and password.

▪NodeName — a unique name of the node in the cluster.

▪User — the admin username used to log in to the Index Server. By default, it is admin.

▪Password — the admin password used to log in to the Index Server. By default, it is admin.

Note: The username and password cannot contain any of the following special characters, and its length cannot exceed 64 characters.
{ } [ ] : " , ' | < > @ & ^ % \ and spaces

▪DataPath — the storage path for all data files of the Index Server.

Note: It is recommended to keep the default path as it is. If you want to modify it, don't use an existing directory and make sure the directory has more than 50GB free space to save all the data files.

▪LogPath — the storage path for all log files of the Index Server.

Note: It is recommended to keep the default path as it is. If you want to modify it, don't use an existing directory and make sure the directory has more than 10GB free space to save all the logs files.

▪BindIp — the IP address of the Index Server.

Note: 0.0.0.0 cannot be used here.

▪Port — the port number that the service of the Index Server listens to. Keep the default value as it is.

▪CPULimit — the maximum CPU utilization that can be consumed by the Index Server. To make both MongoDB Server and Index Server can utilize the CPU resources of the same machine, the recommended value is 35%. And the sum of CPU utilization allocated to the MongoDB Server and Index Server cannot exceed 90% of the machine's CPU.

▪MemoryLimit — the maximum memory capacity of the machine that can be consumed by the Index Server. To make both the MongoDB Server and Index Server can utilize the memory resources of the same machine, the memory allocated to the Index Server must be in the range of 1/8 to1/4 of the physical memory of the machine. If the machine's memory is 16GB, the recommended value is 4GB; if the machine's memory is 32GB, you can modify the value to 8.

Note: The maximum memory that Index Server can utilize is 31GB. Setting the MemoryLimit value higher than 31 will not increase the performance of Index Server. Instead, it may affect the performance of co-existing servers on this machine.

▪UseSSL — whether to enable the encrypted connections to Index Server by using SSL. If yes, replace no with yes.

▪CertAndKeyPath (required only if UseSSL is enabled) — the storage path for all the certification and key files used for SSL authentication.

▪PrivateKey (required only if UseSSL is enabled) — the name of SSL private key file

▪Certificate (required only if UseSSL is enabled) — the name of SSL certificate file containing the public key.

▪CertAuth (required only if UseSSL is enabled) — the name of SSL certificate chain or intermediate certificate (class 2 or class 3 certificate)

▪SingleNode — control whether to enable the cluster deployment. By default, it is yes, which means cluster deployment is disabled. Set its value to no to enable the cluster.

▪ClusterMembers — required only for cluster deployments. Enter the IP addresses of three cluster members.

Note: The first entered cluster member has the tendency to be elected as the master and the master-only node cannot be the first member.

▪MasterOnlyNode — required only for cluster deployments. By default, it is no, which means this node will be used as a master node. It is recommended to set its value to yes only on the Linux server where the arbiter node of the MongoDB replica set is installed.

7)Run the ./install.sh command under the Elasticsearch directory to install the Elasticsearch Server and initialize the username as well as password.

Note: If OpenJDK is installed on your machine, the installation will pause. Enter 3 to install jdk1.8.0_131 and continue the installation.

Note: Don't modify or move the elasticsearch_installed.conf file generated under the /etc/netbrain/ directory. The file is required if you need to re-install the Index Server.

2.Complete the following steps to verify the cluster.

1)Enter http(s)://<IP address of any Elasticsearch node>:<port number>/_cat/nodes?v in the address bar of your web browser. For example, http://10.10.3.142:9200/_cat/nodes?v.

2)Enter the login username and password that you set up in the install_elasticsearch.conf file and click Login to authenticate your credentials. The default credentials are admin/admin.

3)If the login succeeds, you can browse the Index Server information on the login page. This is an example:

The node.role column indicates the role that each node plays in the cluster.

om indicates the node is used as a master-only node.

omdi indicates the node is used as a master node.

oThe * character indicates the location of the current elected master. A cluster can only have one master at any time.

 

See also:

▪Connecting to MongoDB Database