Executing the Upgrade Script
After editing the cluster.config file, execute the smooth upgrade script to upgrade the protector. On all the nodes, the script will:
- Distribute the new parcels.
- Activate the new parcels.
- Removing the old configuration.
- Setting the new configuration.
- Starts the rolling restart to update the required services.
To excute the upgrade script:
- Log in to the Master node.
- Navigate to the directory where the installation files for the new version are extracted.
- To execute the script, run the following command:
./bdp_smooth_upgrade.sh - Press ENTER.
The script upgrades the protector to a newer version using the Rolling Restart feature provided by Cloudera.
'jq' is available. Config loaded: CM_SCHEME = http CM_HOST = <master_node_ip_address> CM_PORT = 7180 CLOUDERA_BASE = /opt/cloudera CLUSTER_NAME = <name_of_the_cluster> BASE_URL = http://<master_node_ip_address>:7180/api CSD_DIR = /opt/cloudera/csd PARCEL_DIR= /opt/cloudera/parcel-repo REMOVE_OLD_PARCELS_AFTER_RR = true REMOVE_PARCEL_STRATEGY = hosts_only Detecting Cloudera Manager API version from http://<master_node_ip_address>:7180/api/version ... Detected API version: v57 CM_URL set to: http://<master_node_ip_address>:7180/api/v57 Checking if cluster '<name_of_the_cluster>' exists in Cloudera Manager... Cluster '<name_of_the_cluster>' exists and is accessible. Checking if cluster-level Rolling Restart is available (non-intrusive)... Rolling Restart appears available (HDFS HA detected: 2xNN, 2xZKFC, 3xJN). Copying files from . to Cloudera directories... Copying JAR files to /opt/cloudera/csd ... Copying parcel files to /opt/cloudera/parcel-repo ... Files copied and permissions set successfully. Extracting parcel versions from .... Detected versions: PTY_BDP : <new_BDP_version>_CDP7.1.p0 PTY_CERT: <new_BDP_version>_CDP7.1.p0 PTY_LOGFORWARDER_CONF: <new_BDP_version>_CDP7.1.p0 Encoded versions: PTY_BDP : <new_BDP_version>_CDP7.1.p0 PTY_CERT: <new_BDP_version>_CDP7.1.p0 PTY_LOGFORWARDER_CONF: <new_BDP_version>_CDP7.1.p0 Pre-upgrade ACTIVE versions from CM: PTY_BDP : <old_BDP_version>_CDP7.1.p0 PTY_CERT : <old_BDP_version>_CDP7.1.p0 PTY_LOGFORWARDER_CONF: <old_BDP_version>_CDP7.1.p0 Restarting Cloudera Manager Server... Cloudera Manager service restart initiated. Waiting for Cloudera Manager API to become available... Cloudera Manager is up and responding. Waiting for PTY_CERT (<new_BDP_version>_CDP7.1.p0) to be recognized by CM ... PTY_CERT recognized. PTY_CERT current stage: ACTIVATED PTY_CERT is already ACTIVATED. Skipping. Waiting for PTY_LOGFORWARDER_CONF (<new_BDP_version>_CDP7.1.p0) to be recognized by CM ... PTY_LOGFORWARDER_CONF recognized. PTY_LOGFORWARDER_CONF current stage: ACTIVATED PTY_LOGFORWARDER_CONF is already ACTIVATED. Skipping. Waiting for PTY_BDP (<new_BDP_version>_CDP7.1.p0) to be recognized by CM ... PTY_BDP recognized. PTY_BDP current stage: ACTIVATED PTY_BDP is already ACTIVATED. Skipping. Running BDP config script (UNSET): /<old_version_dir>/Installation_Files/set_unset_bdp_config.sh Args: --protocol=http:// --cm-server-ip=<master_node_ip_address> --cm-server-port=7180 --cluster-name='<name_of_the_cluster>' --username='<name_of_the_user>' --password=****** --user-choice=UNSET Checking Cluster's existence... Cluster's existence verified. Checking existence of Tez service with name 'tez'. ##O=-# # Service 'tez' exists. Unsetting Tez's config... ##################################################################################### 100.0% Tez Service wide config ('tez.cluster.additional.classpath.prefix') has been updated. Checking existence of Impala service with name 'impala'. Service 'impala' exists. Unsetting Impala's config... ##################################################################################### 100.0% Impala's 'IMPALAD_role_env_safety_valve' config for Role Group 'impala-IMPALAD-2' has been updated. ##O=-# # ##################################################################################### 100.0% Impala's 'IMPALAD_role_env_safety_valve' config for Role Group 'impala-IMPALAD-1' has been updated. ##O=-# # ##################################################################################### 100.0% Impala's 'IMPALAD_role_env_safety_valve' config for Role Group 'impala-IMPALAD-BASE' has been updated. Checking existence of Spark on Yarn service with name 'spark_on_yarn'. Service 'spark_on_yarn' exists. Unsetting Spark on Yarn's config... ##################################################################################### 100.0% Spark on Yarn Service wide config ('spark-conf/spark-env.sh_service_safety_valve') has been updated. Running BDP config script (SET): ./set_unset_bdp_config.sh Args: --protocol=http:// --cm-server-ip=<master_node_ip_address> --cm-server-port=7180 --cluster-name='<name_of_the_cluster>' --username='<name_of_the_user>' --password=****** --user-choice=SET Checking Cluster's existence... Cluster's existence verified. Checking existence of Tez service with name 'tez'. ##O=-# # Service 'tez' exists. Setting Tez's config... ##O=-# # ##################################################################################### 100.0% Tez Service wide config ('tez.cluster.additional.classpath.prefix') has been updated. Checking existence of Impala service with name 'impala'. Service 'impala' exists. Setting Impala's config... ##################################################################################### 100.0% Impala's 'IMPALAD_role_env_safety_valve' config for Role Group 'impala-IMPALAD-2' has been updated. ##################################################################################### 100.0% Impala's 'IMPALAD_role_env_safety_valve' config for Role Group 'impala-IMPALAD-1' has been updated. ##################################################################################### 100.0% Impala's 'IMPALAD_role_env_safety_valve' config for Role Group 'impala-IMPALAD-BASE' has been updated. Checking existence of Spark on Yarn service with name 'spark_on_yarn'. Service 'spark_on_yarn' exists. Setting Spark on Yarn's config... ##################################################################################### 100.0% Spark on Yarn Service wide config ('spark-conf/spark-env.sh_service_safety_valve') has been updated. ROLLING_EXCLUDE_SERVICES is set. Using restartServiceNames API to exclude: impala Waiting for CM command id=<command_ID> to complete ... - RollingRestart progress: 0%, active: ?, success: true Command <command_ID> finished successfully. Rolling restart finished successfully. Evaluating convergence before parcel cleanup ... Warning: Permanently added 'edge.localdomain.com' (ECDSA) to the list of known hosts. Warning: Permanently added 'master.localdomain.com' (ECDSA) to the list of known hosts. Warning: Permanently added 'node1.localdomain.com' (ECDSA) to the list of known hosts. Warning: Permanently added 'node2.localdomain.com' (ECDSA) to the list of known hosts. Warning: Permanently added 'node3.localdomain.com' (ECDSA) to the list of known hosts. Cluster appears converged: all hosts use PTY_BDP <new_BDP_version>_CDP7.1.p0 and no old-parcel processes found. Converged ? cleaning old parcels (REMOVE_OLD_PARCELS_AFTER_RR=true) ... Selected PTY_CERT old version to clean: Discovering previous parcel versions in: <old_version_dir>/Installation_Files Previous PTY_BDP version: <old_BDP_version>_CDP7.1.p0 Previous PTY_CERT version: <old_BDP_version>_CDP7.1.p0 <old_BDP_version>_CDP7.1.p0 Cleaning old parcel PTY_CERT (Discovering previous parcel versions in: <old_version_dir>/Installation_Files Previous PTY_BDP version: <old_BDP_version>_CDP7.1.p0 Previous PTY_CERT version: <old_BDP_version>_CDP7.1.p0 <old_BDP_version>_CDP7.1.p0) ... Current stage: DISTRIBUTED Removing distribution of PTY_CERT Discovering previous parcel versions in: <old_version_dir>/Installation_Files Previous PTY_BDP version: <old_BDP_version>_CDP7.1.p0 Previous PTY_CERT version: <old_BDP_version>_CDP7.1.p0 <old_BDP_version>_CDP7.1.p0 from hosts ... Waiting for PTY_CERT to reach stage: DOWNLOADED Current stage for PTY_CERT: UNDISTRIBUTING Current stage for PTY_CERT: UNDISTRIBUTING Current stage for PTY_CERT: DOWNLOADED Done with PTY_CERT (Discovering previous parcel versions in: <old_version_dir>/Installation_Files Previous PTY_BDP version: <old_BDP_version>_CDP7.1.p0 Previous PTY_CERT version: <old_BDP_version>_CDP7.1.p0 <old_BDP_version>_CDP7.1.p0). Selected PTY_BDP old version to clean: Discovering previous parcel versions in: <old_version_dir>/Installation_Files Previous PTY_BDP version: <old_BDP_version>_CDP7.1.p0 Previous PTY_CERT version: <old_BDP_version>_CDP7.1.p0 <old_BDP_version>_CDP7.1.p0 Cleaning old parcel PTY_BDP (Discovering previous parcel versions in: <old_version_dir>/Installation_Files Previous PTY_BDP version: <old_BDP_version>_CDP7.1.p0 Previous PTY_CERT version: <old_BDP_version>_CDP7.1.p0 <old_BDP_version>_CDP7.1.p0) ... Current stage: DISTRIBUTED Removing distribution of PTY_BDP Discovering previous parcel versions in: <old_version_dir>/Installation_Files Previous PTY_BDP version: <old_BDP_version>_CDP7.1.p0 Previous PTY_CERT version: <old_BDP_version>_CDP7.1.p0 <old_BDP_version>_CDP7.1.p0 from hosts ... Waiting for PTY_BDP to reach stage: DOWNLOADED Current stage for PTY_BDP: UNDISTRIBUTING Current stage for PTY_BDP: UNDISTRIBUTING Current stage for PTY_BDP: DOWNLOADED Done with PTY_BDP (Discovering previous parcel versions in: <old_version_dir>/Installation_Files Previous PTY_BDP version: <old_BDP_version>_CDP7.1.p0 Previous PTY_CERT version: <old_BDP_version>_CDP7.1.p0 <old_BDP_version>_CDP7.1.p0). Old parcels cleanup completed.
Feedback
Was this page helpful?