Creating a Cluster

The procedures mentioned in this section are applicable only for the Bootstrap approach to install the Big Data Protector.

Perform the following steps to create an EMR cluster on AWS and install Big Data Protector on all the nodes in the EMR cluster.

To install Big Data Protector on a New EMR Cluster:

1. On the AWS services screen, click EMR under the Analytics section.

   The Amazon EMR screen appears.

2. Click Create cluster.

   The Create Cluster - Quick Options screen appears.

3. Type the name of the cluster in the Cluster name box.

4. Depending on the requirements, enter the sum of the master and core nodes in the Number of instances box.

5. Click Create cluster.

   The Software and Steps tab on the Create Cluster - Advanced Options screen appears.

6. Depending on the requirements, select the components under the Software Configuration section.

7. Click Next.

   The Hardware tab on the Create Cluster - Advanced Options screen appears.

8. On the Hardware tab, if required, you can add or reduce the number of instances of the Master, Core, and Task nodes.

9. Click Next.

   The General Cluster Settings tab on the Create Cluster - Advanced Options screen appears.

10. Type the name of the cluster in the Cluster name box.

11. Under the Bootstrap Actions area, in the Add bootstrap action drop-down list, click Custom action.

    The Add Bootstrap Action dialog box appears.

12. Enter the name of the bootstrap action in the Name box.

13. To select the location of the bootstrap script, click the icon besides the Script location box.

    The Select S3 File dialog box appears.

14. Enter the path of the S3 bucket in the URL box.

    The contents of the S3 bucket appear.

15. Select the bdp_bootstrap_installer.sh file from the S3 bucket.

16. Click Select.

    The Big Data Protector bootstrap script file is selected and the Add Bootstrap Action dialog box appears.

17. To specify the directory in which the Big Data Protector needs to be installed on the nodes in the cluster, then provide the directory path in the Optional arguments box.

    If an installation directory for the Big Data Protector is not specified, then /opt/protegrity/ is considered as the default directory.

18. Click Add.

    The General Cluster Settings tab on the Create Cluster - Advanced Options screen appears and the Bootstrap actions are updated.

19. Click Next.

    The Security tab on the Create Cluster - Advanced Options screen appears.

20. Select the required EC2 key pair for the EMR cluster from the EC2 key pair drop-down list.

21. Click Create Cluster.

    The EMR cluster is created, Big Data Protector is installed on all the nodes in the cluster, and the required Big Data Protector parameters are configured.

22. You can also install create a new EMR cluster and install Big Data Protector on the nodes in the cluster using the CLI using the following command:

    aws emr create-cluster --auto-scaling-role EMR_AutoScaling_DefaultRole --termination-protected --applications Name=Hadoop Name=Hive Name=Pig Name=Hue Name=Spark Name=Tez Name=HBase --bootstrap-actions '[{"Path":"<S3_Path_For_BootstrapInstaller>","Name":"<Script_Name>"}]' --ec2-attributes '{"KeyName":"<KEY_NAME>","InstanceProfile":"EMR_EC2_DefaultRole","EmrManagedSlaveSecurityGroup":"sg-c8ef00de","EmrManagedMasterSecurityGroup":"sg-2deb043b"}' --service-role EMR_DefaultRole --enable-debugging --release-label emr-<EMR_Version> --log-uri 's3n://aws-logs-406396743807-us-east-1/elasticmapreduce/' --name '<Cluster_Name>' --instance-groups '[{"InstanceCount":2,"InstanceGroupType":"CORE","InstanceType":"m3.xlarge","Name":"Core - 2"},{"InstanceCount":1,"InstanceGroupType":"MASTER","InstanceType":"m3.xlarge","Name":"Master - 1"}]' –
    scale-down-behavior TERMINATE_AT_INSTANCE_HOUR --region us-east-1
    

    where:

    • S3_Path_For_BootstrapInstaller: Specifies the S3 bucket path containing the Big Data Protector bootstrap installer script.
    • Script_Name: Specifies the name of the Big Data Protector installation script.
    • KEY_NAME: Specifies the Private Key file on the Master node in the EMR cluster, which is used to communicate with the other nodes in the cluster.
    • Cluster_Name: Specifies the name of the new EMR cluster.