Installing the Protector on all the Nodes
Installing the Protector on all the Nodes using the Static Installer
The steps mentioned in this section are applicable only for the Static Installer approach to install the Big Data Protector.
Log in to the Master or Lead node of the EMR cluster.
Navigate to the directory that contains the
BdpInstallx.x.x_Linux_<BDP_version>.shscript.To run the installer, execute the following script:
./BdpInstallx.x.x_Linux_<BDP_version>.shPress ENTER.
The prompt to continue the installation of the Big Data Protector appears.
************************************************************************************ Welcome to the Hadoop Big Data Protector Setup Wizard ************************************************************************************ This will install the Hadoop Big Data Protector on your system. This installation requires a Private Key file for communicating with other nodes in the cluster. Do you want to continue? [yes or no]:To continue, type
yes.Press ENTER.
The prompt to enter path of the Private Key file (.pem file) appears.
Big Data Protector installation started Enter the path of the Private Key (.PEM) file:Enter the path of the
.PEMfile.Press ENTER.
The prompt to enter ESA hostname or IP address appears.
libhadoop.so located in directory '/usr/lib/hadoop/lib/native' Unpacking... Extracting files... Preparing for cluster deploy, Wait... Enter ESA Hostname or IP Address:If you have installed a proxy, then enter the IP address of the proxy node. Alternatively, enter the IP Address of ESA.
Press ENTER.
The prompt to enter the listening port for ESA appears.
Enter ESA host listening port [8443]:Enter the port for ESA.
Press ENTER.
The prompt to enter the JWT token appears.
If you have an existing ESA JSON Web Token (JWT) with Export Certificates role, enter it otherwise enter 'no':Enter the JWT token.
Press ENTER.
If you fail to provide a JWT token, the script will prompt to enter the username and password for ESA.
JWT was not provided. Script will now prompt for ESA username and password. Enter ESA Username:Enter the username for ESA.
Press ENTER.
The prompt to enter the password appears.
************************************************************************************ Welcome to the RPAgent Setup Wizard. ************************************************************************************ Unpacking................... Extracting files... Unpacked rpagent compressed file... RPAgent Installing in Lead Node... Please enter the password for downloading certificates[]:Enter the password.
Press ENTER.
The script retrieves the JWT token from ESA, installs the RPAgent, and the prompt to select the Audit Store type appears.
Unpacking... Extracting files... Obtaining token from <ESA_IP_Address>:8443... Downloading certificates from <ESA_IP_Address>:8443... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 11264 100 11264 0 0 12124 0 --:--:-- --:--:-- --:--:-- 12111 Extracting certificates... Certificates successfully downloaded and stored in /opt/protegrity/rpagent/data Protegrity RPAgent installed in /opt/protegrity/rpagent. RPAgent installed on Lead node at location /opt/protegrity/rpagent. Performing install on other nodes... RPAgent installed on other nodes at location /opt/protegrity/rpagent. Check the status in /opt/protegrity/logs/rpagent_setup.log Select the Audit Store type where Log Forwarder(s) should send logs to. [ 1 ] : Protegrity Audit Store [ 2 ] : External Audit Store [ 3 ] : Protegrity Audit Store + External Audit Store Enter the no.:Depending on the Audit Store type, select any one of the following options:
Option Description 1To use the default setting using the Protegrity Audit Store appliance, type 1. If you enter1, then the default Fluent Bit configuration files are used and Fluent Bit will forward the logs to the Protegrity Audit Store appliances.2To use an external audit store, type 2. If you enter2, then the default Fluent Bit configuration files used for the External Audit Store (out.conf and upstream.cfg in the/opt/protegrity/fluent-bit/data/config.d/directory) are renamed (out.conf.bkp and upstream.cfg.bkp) so that they will not be used by Fluent Bit. Additionally, the custom Fluent Bit configuration files for the external audit store are copied to the /opt/protegrity/fluent-bit/data/config.d/ directory.3To use a combination of the default setting with an external audit store, type 3. If you enter3, then the default Fluent Bit configuration files used for the Protegrity Audit Store (out.conf and upstream.cfg in the/opt/protegrity/fluent-bit/data/config.d/directory) are not renamed. However, the custom Fluent Bit configuration files for the external audit store are copied to the/opt/protegrity/fluent-bit/data/config.d/directory.Press ENTER.
The prompt to enter the comma separated list of hostnames/IP addresses appears.
Enter comma-separated list of Hostnames/IP Addresses and/or Ports of Protegrity Audit Store. Allowed Syntax: hostname[:port][,hostname[:port],hostname[:port]...] (Default Value - <ESA_IP_Address>:9200) Enter the list:To use the default value, press ENTER.
The prompt to enter the location of the Fluent Bit configuration file appears.
Enter the local directory path on this node that stores the custom Fluent-Bit configuration files for External Audit Store:Note: The script will display this prompt only if you select option
2in step19. When you select option2in step 19, the custom configuration files are copied to the/<Installation directory>/fluent-bit/data/config.d/directory on all the EMR nodes selected for installation.Enter the path that contains the Fluent Bit configuration file.
Press ENTER.
The prompt to save the RPAgent’s log in a file appears.
Do you want RPAgent's log to be generated in a file? [yes or no]:To generate the logs in a file, type
yes.Press ENTER.
The script installs the protector on all the nodes in the cluster.
RPAgent's log will be generated in a file. ************************************************************************************ Welcome to the LogForwarder Setup Wizard. ************************************************************************************ Unpacking................... Extracting files... Unpacked logforwarder compressed file... Logforwarder Installing in Lead Node... Unpacking... Extracting files... Protegrity Log Forwarder installed in /opt/protegrity/logforwarder. LogForwarder installed on Lead node at location /opt/protegrity/logforwarder. Performing install on other nodes... Logforwarder installed on other nodes at location /opt/protegrity/logforwarder. Check the status in /opt/protegrity/logs/logforwarder_setup.log ************************************************************************************ Welcome to the JcoreLite Setup Wizard. ************************************************************************************ Unpacking................... Extracting files... Unpacked jcorelite compressed file... Installing JcoreLite .... JcoreLite installed on lead node at location /opt/protegrity/bdp/lib. Performing install on other nodes... JcoreLite installed on other nodes at location /opt/protegrity/bdp/lib. Check the status in /opt/protegrity/logs/jcorelite_setup.log ************************************************************************************ Welcome to the Hive Protector Setup Wizard. ************************************************************************************ Unpacking................... Extracting files... Unpacked pephive compressed file... Hive Big Data Protector installed on lead node at location /opt/protegrity/bdp/lib/ and /opt/protegrity/pephive/scripts/. Performing install on other nodes... Hive Big Data Protector installed on other nodes at location /opt/protegrity/bdp/lib/ and /opt/protegrity/pephive/scripts/. Check the status in /opt/protegrity/logs/pephive_setup.log ************************************************************************************ Welcome to the Pig Protector Setup Wizard. ************************************************************************************ Unpacking................... Extracting files... Unpacked peppig compressed file... Pig Big Data Protector installed on lead node at location /opt/protegrity/bdp/lib/ and /opt/protegrity/peppig. Performing install on other nodes... Pig Big Data Protector installed on other nodes at location /opt/protegrity/bdp/lib/ and /opt/protegrity/peppig. Check the status in /opt/protegrity/logs/peppig_setup.log ************************************************************************************ Welcome to the MapReduce Protector Setup Wizard. ************************************************************************************ Unpacking................... Extracting files... Unpacked pepmapreduce compressed file... Mapreduce Big Data Protector installed on lead node at location /opt/protegrity/bdp/lib/. Performing install on other nodes... Mapreduce Big Data Protector installed on other nodes at location /opt/protegrity/bdp/lib/. Check the status in /opt/protegrity/logs/pepmapreduce_setup.log ************************************************************************************ Welcome to the Hbase Protector Setup Wizard. ************************************************************************************ Unpacking................... Extracting files... Unpacked pephbase compressed file... Hbase Big Data Protector installed on lead node at location /opt/protegrity/bdp/lib/. Performing install on other nodes... Hbase Big Data Protector installed on other nodes at location /opt/protegrity/bdp/lib/. Check the status in /opt/protegrity/logs/pephbase_setup.log ************************************************************************************ Welcome to the Spark Protector Setup Wizard. ************************************************************************************ Unpacking................... Extracting files... Unpacked pepspark compressed file... Spark Big Data Protector installed on lead node at location /opt/protegrity/bdp/lib/ and /opt/protegrity/pepspark/scripts/. Performing install on other nodes... Spark Big Data Protector installed on other nodes at location /opt/protegrity/bdp/lib/ and /opt/protegrity/pepspark/scripts/. Check the status in /opt/protegrity/logs/pepspark_setup.log Starting Logforwarder on lead node... Starting Logforwarder on other nodes... Starting RPAgent on lead node... Starting RPAgent on other nodes... Hadoop Big Data Protector installed in /opt/protegrity. Generating Big Data Protector installation status report ... Clearing previous logs files ... Installation Status report generated in /opt/protegrity/cluster_utils/installation_report.txtRestart the Hadoop, Hive, and HBase service daemon processes to start using the updated configuration.
Feedback
Was this page helpful?