Installing the UDFs using the Helper Script
The Big Data Protector build provides helper scripts to register the user-defined functions for the following components:
- Hive
- Impala
- SparkSQL
- PySpark
To use the helper scripts to drop the UDFs, refer Drop the UDFs using the Helper Script.
Registering the Termporary Hive UDFs
Log in to the master node with a user account having permissions to create and drop UDFs.
To navigate to the directory that contains the helper script, run the following command:
cd /opt/cloudera/parcels/PTY_BDP/pephive/scriptsTo create the UDFs using the helper script, run the following command:
beeline -f create_temp_hive_udfs.hql;Execute the command in beeline after establishing a connection.
Press ENTER.
The script creates all the temporary user-defined functions for Hive.
Connected to: Apache Hive (version 3.1.3000.7.3.1.400-100) Driver: Hive JDBC (version 3.1.3000.7.3.1.400-100) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyGetVersion AS 'com.protegrity.hive.udf.ptyGetVersion'; INFO : Compiling command(queryId=hive_20250916121741_49e4a9e3-5322-45b1-bc74-b812ae853934): CREATE TEMPORARY FUNCTION ptyGetVersion AS 'com.protegrity.hive.udf.ptyGetVersion' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121741_49e4a9e3-5322-45b1-bc74-b812ae853934); Time taken: 0.088 seconds INFO : Executing command(queryId=hive_20250916121741_49e4a9e3-5322-45b1-bc74-b812ae853934): CREATE TEMPORARY FUNCTION ptyGetVersion AS 'com.protegrity.hive.udf.ptyGetVersion' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121741_49e4a9e3-5322-45b1-bc74-b812ae853934); Time taken: 0.003 seconds INFO : OK No rows affected (0.154 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyGetVersionExtended AS 'com.protegrity.hive.udf.ptyGetVersionExtended'; INFO : Compiling command(queryId=hive_20250916121741_2a7a85d4-b1b6-479c-8552-d26f1cce1d53): CREATE TEMPORARY FUNCTION ptyGetVersionExtended AS 'com.protegrity.hive.udf.ptyGetVersionExtended' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121741_2a7a85d4-b1b6-479c-8552-d26f1cce1d53); Time taken: 0.02 seconds INFO : Executing command(queryId=hive_20250916121741_2a7a85d4-b1b6-479c-8552-d26f1cce1d53): CREATE TEMPORARY FUNCTION ptyGetVersionExtended AS 'com.protegrity.hive.udf.ptyGetVersionExtended' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121741_2a7a85d4-b1b6-479c-8552-d26f1cce1d53); Time taken: 0.002 seconds INFO : OK No rows affected (0.052 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyWhoAmI AS 'com.protegrity.hive.udf.ptyWhoAmI'; INFO : Compiling command(queryId=hive_20250916121741_ebe06a7c-c265-4705-ae7b-8181065d1c8e): CREATE TEMPORARY FUNCTION ptyWhoAmI AS 'com.protegrity.hive.udf.ptyWhoAmI' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121741_ebe06a7c-c265-4705-ae7b-8181065d1c8e); Time taken: 0.021 seconds INFO : Executing command(queryId=hive_20250916121741_ebe06a7c-c265-4705-ae7b-8181065d1c8e): CREATE TEMPORARY FUNCTION ptyWhoAmI AS 'com.protegrity.hive.udf.ptyWhoAmI' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121741_ebe06a7c-c265-4705-ae7b-8181065d1c8e); Time taken: 0.001 seconds INFO : OK No rows affected (0.05 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyProtectStr AS 'com.protegrity.hive.udf.ptyProtectStr'; INFO : Compiling command(queryId=hive_20250916121741_260aef88-ab60-4d0b-b502-ebf178bd9065): CREATE TEMPORARY FUNCTION ptyProtectStr AS 'com.protegrity.hive.udf.ptyProtectStr' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121741_260aef88-ab60-4d0b-b502-ebf178bd9065); Time taken: 0.021 seconds INFO : Executing command(queryId=hive_20250916121741_260aef88-ab60-4d0b-b502-ebf178bd9065): CREATE TEMPORARY FUNCTION ptyProtectStr AS 'com.protegrity.hive.udf.ptyProtectStr' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121741_260aef88-ab60-4d0b-b502-ebf178bd9065); Time taken: 0.001 seconds INFO : OK No rows affected (0.052 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyUnprotectStr AS 'com.protegrity.hive.udf.ptyUnprotectStr'; INFO : Compiling command(queryId=hive_20250916121741_feba856d-e516-4b08-805a-3ee89aeff524): CREATE TEMPORARY FUNCTION ptyUnprotectStr AS 'com.protegrity.hive.udf.ptyUnprotectStr' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121741_feba856d-e516-4b08-805a-3ee89aeff524); Time taken: 0.021 seconds INFO : Executing command(queryId=hive_20250916121741_feba856d-e516-4b08-805a-3ee89aeff524): CREATE TEMPORARY FUNCTION ptyUnprotectStr AS 'com.protegrity.hive.udf.ptyUnprotectStr' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121741_feba856d-e516-4b08-805a-3ee89aeff524); Time taken: 0.001 seconds INFO : OK No rows affected (0.053 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyReprotect AS 'com.protegrity.hive.udf.ptyReprotect'; INFO : Compiling command(queryId=hive_20250916121741_cd516b0a-648a-4893-92f0-7486ee4525d9): CREATE TEMPORARY FUNCTION ptyReprotect AS 'com.protegrity.hive.udf.ptyReprotect' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121741_cd516b0a-648a-4893-92f0-7486ee4525d9); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121741_cd516b0a-648a-4893-92f0-7486ee4525d9): CREATE TEMPORARY FUNCTION ptyReprotect AS 'com.protegrity.hive.udf.ptyReprotect' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121741_cd516b0a-648a-4893-92f0-7486ee4525d9); Time taken: 0.001 seconds INFO : OK No rows affected (0.049 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyProtectUnicode AS 'com.protegrity.hive.udf.ptyProtectUnicode'; INFO : Compiling command(queryId=hive_20250916121741_ab35e17d-4050-4ae9-a3eb-a796be96f739): CREATE TEMPORARY FUNCTION ptyProtectUnicode AS 'com.protegrity.hive.udf.ptyProtectUnicode' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121741_ab35e17d-4050-4ae9-a3eb-a796be96f739); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121741_ab35e17d-4050-4ae9-a3eb-a796be96f739): CREATE TEMPORARY FUNCTION ptyProtectUnicode AS 'com.protegrity.hive.udf.ptyProtectUnicode' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121741_ab35e17d-4050-4ae9-a3eb-a796be96f739); Time taken: 0.001 seconds INFO : OK No rows affected (0.048 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyUnprotectUnicode AS 'com.protegrity.hive.udf.ptyUnprotectUnicode'; INFO : Compiling command(queryId=hive_20250916121741_10e20116-c909-45e3-be3d-870ce13ef54b): CREATE TEMPORARY FUNCTION ptyUnprotectUnicode AS 'com.protegrity.hive.udf.ptyUnprotectUnicode' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121741_10e20116-c909-45e3-be3d-870ce13ef54b); Time taken: 0.02 seconds INFO : Executing command(queryId=hive_20250916121741_10e20116-c909-45e3-be3d-870ce13ef54b): CREATE TEMPORARY FUNCTION ptyUnprotectUnicode AS 'com.protegrity.hive.udf.ptyUnprotectUnicode' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121741_10e20116-c909-45e3-be3d-870ce13ef54b); Time taken: 0.001 seconds INFO : OK No rows affected (0.048 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyReprotectUnicode AS 'com.protegrity.hive.udf.ptyReprotectUnicode'; INFO : Compiling command(queryId=hive_20250916121741_7a142263-62c2-44b1-bbb0-05530e77daa7): CREATE TEMPORARY FUNCTION ptyReprotectUnicode AS 'com.protegrity.hive.udf.ptyReprotectUnicode' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121741_7a142263-62c2-44b1-bbb0-05530e77daa7); Time taken: 0.021 seconds INFO : Executing command(queryId=hive_20250916121741_7a142263-62c2-44b1-bbb0-05530e77daa7): CREATE TEMPORARY FUNCTION ptyReprotectUnicode AS 'com.protegrity.hive.udf.ptyReprotectUnicode' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121741_7a142263-62c2-44b1-bbb0-05530e77daa7); Time taken: 0.001 seconds INFO : OK No rows affected (0.056 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyProtectShort AS 'com.protegrity.hive.udf.ptyProtectShort'; INFO : Compiling command(queryId=hive_20250916121742_1b5414e1-175e-4889-bad3-e64ca6595fe0): CREATE TEMPORARY FUNCTION ptyProtectShort AS 'com.protegrity.hive.udf.ptyProtectShort' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_1b5414e1-175e-4889-bad3-e64ca6595fe0); Time taken: 0.022 seconds INFO : Executing command(queryId=hive_20250916121742_1b5414e1-175e-4889-bad3-e64ca6595fe0): CREATE TEMPORARY FUNCTION ptyProtectShort AS 'com.protegrity.hive.udf.ptyProtectShort' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_1b5414e1-175e-4889-bad3-e64ca6595fe0); Time taken: 0.001 seconds INFO : OK No rows affected (0.053 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyUnprotectShort AS 'com.protegrity.hive.udf.ptyUnprotectShort'; INFO : Compiling command(queryId=hive_20250916121742_58b96d4d-25a2-4886-a4c5-ad1c69164016): CREATE TEMPORARY FUNCTION ptyUnprotectShort AS 'com.protegrity.hive.udf.ptyUnprotectShort' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_58b96d4d-25a2-4886-a4c5-ad1c69164016); Time taken: 0.026 seconds INFO : Executing command(queryId=hive_20250916121742_58b96d4d-25a2-4886-a4c5-ad1c69164016): CREATE TEMPORARY FUNCTION ptyUnprotectShort AS 'com.protegrity.hive.udf.ptyUnprotectShort' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_58b96d4d-25a2-4886-a4c5-ad1c69164016); Time taken: 0.001 seconds INFO : OK No rows affected (0.058 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyProtectInt AS 'com.protegrity.hive.udf.ptyProtectInt'; INFO : Compiling command(queryId=hive_20250916121742_8e53692d-4094-4b1a-98fa-829d90b3dea3): CREATE TEMPORARY FUNCTION ptyProtectInt AS 'com.protegrity.hive.udf.ptyProtectInt' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_8e53692d-4094-4b1a-98fa-829d90b3dea3); Time taken: 0.023 seconds INFO : Executing command(queryId=hive_20250916121742_8e53692d-4094-4b1a-98fa-829d90b3dea3): CREATE TEMPORARY FUNCTION ptyProtectInt AS 'com.protegrity.hive.udf.ptyProtectInt' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_8e53692d-4094-4b1a-98fa-829d90b3dea3); Time taken: 0.001 seconds INFO : OK No rows affected (0.058 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyUnprotectInt AS 'com.protegrity.hive.udf.ptyUnprotectInt'; INFO : Compiling command(queryId=hive_20250916121742_7762d5cd-1934-45d4-8297-d76236575744): CREATE TEMPORARY FUNCTION ptyUnprotectInt AS 'com.protegrity.hive.udf.ptyUnprotectInt' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_7762d5cd-1934-45d4-8297-d76236575744); Time taken: 0.02 seconds INFO : Executing command(queryId=hive_20250916121742_7762d5cd-1934-45d4-8297-d76236575744): CREATE TEMPORARY FUNCTION ptyUnprotectInt AS 'com.protegrity.hive.udf.ptyUnprotectInt' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_7762d5cd-1934-45d4-8297-d76236575744); Time taken: 0.001 seconds INFO : OK No rows affected (0.048 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyProtectBigInt as 'com.protegrity.hive.udf.ptyProtectBigInt'; INFO : Compiling command(queryId=hive_20250916121742_7fb43101-562a-441b-8acb-decbe8e216ea): CREATE TEMPORARY FUNCTION ptyProtectBigInt as 'com.protegrity.hive.udf.ptyProtectBigInt' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_7fb43101-562a-441b-8acb-decbe8e216ea); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121742_7fb43101-562a-441b-8acb-decbe8e216ea): CREATE TEMPORARY FUNCTION ptyProtectBigInt as 'com.protegrity.hive.udf.ptyProtectBigInt' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_7fb43101-562a-441b-8acb-decbe8e216ea); Time taken: 0.001 seconds INFO : OK No rows affected (0.046 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyUnprotectBigInt as 'com.protegrity.hive.udf.ptyUnprotectBigInt'; INFO : Compiling command(queryId=hive_20250916121742_1f107ea2-c7b1-4d18-8f33-04442d84b80e): CREATE TEMPORARY FUNCTION ptyUnprotectBigInt as 'com.protegrity.hive.udf.ptyUnprotectBigInt' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_1f107ea2-c7b1-4d18-8f33-04442d84b80e); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121742_1f107ea2-c7b1-4d18-8f33-04442d84b80e): CREATE TEMPORARY FUNCTION ptyUnprotectBigInt as 'com.protegrity.hive.udf.ptyUnprotectBigInt' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_1f107ea2-c7b1-4d18-8f33-04442d84b80e); Time taken: 0.001 seconds INFO : OK No rows affected (0.047 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyProtectFloat as 'com.protegrity.hive.udf.ptyProtectFloat'; INFO : Compiling command(queryId=hive_20250916121742_88d8ed50-72d1-4e4b-9c9e-ed105e099c7b): CREATE TEMPORARY FUNCTION ptyProtectFloat as 'com.protegrity.hive.udf.ptyProtectFloat' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_88d8ed50-72d1-4e4b-9c9e-ed105e099c7b); Time taken: 0.02 seconds INFO : Executing command(queryId=hive_20250916121742_88d8ed50-72d1-4e4b-9c9e-ed105e099c7b): CREATE TEMPORARY FUNCTION ptyProtectFloat as 'com.protegrity.hive.udf.ptyProtectFloat' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_88d8ed50-72d1-4e4b-9c9e-ed105e099c7b); Time taken: 0.001 seconds INFO : OK No rows affected (0.047 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyUnprotectFloat as 'com.protegrity.hive.udf.ptyProtectFloat'; INFO : Compiling command(queryId=hive_20250916121742_7455402f-bc13-425b-98b0-204c72b088c2): CREATE TEMPORARY FUNCTION ptyUnprotectFloat as 'com.protegrity.hive.udf.ptyProtectFloat' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_7455402f-bc13-425b-98b0-204c72b088c2); Time taken: 0.02 seconds INFO : Executing command(queryId=hive_20250916121742_7455402f-bc13-425b-98b0-204c72b088c2): CREATE TEMPORARY FUNCTION ptyUnprotectFloat as 'com.protegrity.hive.udf.ptyProtectFloat' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_7455402f-bc13-425b-98b0-204c72b088c2); Time taken: 0.001 seconds INFO : OK No rows affected (0.046 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyProtectDouble as 'com.protegrity.hive.udf.ptyProtectDouble'; INFO : Compiling command(queryId=hive_20250916121742_0f1a9810-1311-4dcd-a4c6-bfe52088cf1b): CREATE TEMPORARY FUNCTION ptyProtectDouble as 'com.protegrity.hive.udf.ptyProtectDouble' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_0f1a9810-1311-4dcd-a4c6-bfe52088cf1b); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121742_0f1a9810-1311-4dcd-a4c6-bfe52088cf1b): CREATE TEMPORARY FUNCTION ptyProtectDouble as 'com.protegrity.hive.udf.ptyProtectDouble' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_0f1a9810-1311-4dcd-a4c6-bfe52088cf1b); Time taken: 0.001 seconds INFO : OK No rows affected (0.048 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyUnprotectDouble as 'com.protegrity.hive.udf.ptyUnprotectDouble'; INFO : Compiling command(queryId=hive_20250916121742_cab992d8-efa2-41e6-b83e-71074d444565): CREATE TEMPORARY FUNCTION ptyUnprotectDouble as 'com.protegrity.hive.udf.ptyUnprotectDouble' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_cab992d8-efa2-41e6-b83e-71074d444565); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121742_cab992d8-efa2-41e6-b83e-71074d444565): CREATE TEMPORARY FUNCTION ptyUnprotectDouble as 'com.protegrity.hive.udf.ptyUnprotectDouble' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_cab992d8-efa2-41e6-b83e-71074d444565); Time taken: 0.001 seconds INFO : OK No rows affected (0.046 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyProtectDec as 'com.protegrity.hive.udf.ptyProtectDec'; INFO : Compiling command(queryId=hive_20250916121742_95374cee-053b-42b1-a7f9-39d4d0eb24c6): CREATE TEMPORARY FUNCTION ptyProtectDec as 'com.protegrity.hive.udf.ptyProtectDec' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_95374cee-053b-42b1-a7f9-39d4d0eb24c6); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121742_95374cee-053b-42b1-a7f9-39d4d0eb24c6): CREATE TEMPORARY FUNCTION ptyProtectDec as 'com.protegrity.hive.udf.ptyProtectDec' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_95374cee-053b-42b1-a7f9-39d4d0eb24c6); Time taken: 0.001 seconds INFO : OK No rows affected (0.046 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyUnprotectDec as 'com.protegrity.hive.udf.ptyUnprotectDec'; INFO : Compiling command(queryId=hive_20250916121742_1d045781-69d8-4a55-a87b-0f43121b2a09): CREATE TEMPORARY FUNCTION ptyUnprotectDec as 'com.protegrity.hive.udf.ptyUnprotectDec' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_1d045781-69d8-4a55-a87b-0f43121b2a09); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121742_1d045781-69d8-4a55-a87b-0f43121b2a09): CREATE TEMPORARY FUNCTION ptyUnprotectDec as 'com.protegrity.hive.udf.ptyUnprotectDec' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_1d045781-69d8-4a55-a87b-0f43121b2a09); Time taken: 0.001 seconds INFO : OK No rows affected (0.046 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyProtectHiveDecimal as 'com.protegrity.hive.udf.ptyProtectHiveDecimal'; INFO : Compiling command(queryId=hive_20250916121742_281198df-c918-470f-99a5-bca98a5936b7): CREATE TEMPORARY FUNCTION ptyProtectHiveDecimal as 'com.protegrity.hive.udf.ptyProtectHiveDecimal' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_281198df-c918-470f-99a5-bca98a5936b7); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121742_281198df-c918-470f-99a5-bca98a5936b7): CREATE TEMPORARY FUNCTION ptyProtectHiveDecimal as 'com.protegrity.hive.udf.ptyProtectHiveDecimal' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_281198df-c918-470f-99a5-bca98a5936b7); Time taken: 0.001 seconds INFO : OK No rows affected (0.046 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyUnprotectHiveDecimal as 'com.protegrity.hive.udf.ptyUnprotectHiveDecimal'; INFO : Compiling command(queryId=hive_20250916121742_b3d564e5-22fa-4985-b988-eac4d188b3fa): CREATE TEMPORARY FUNCTION ptyUnprotectHiveDecimal as 'com.protegrity.hive.udf.ptyUnprotectHiveDecimal' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_b3d564e5-22fa-4985-b988-eac4d188b3fa); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121742_b3d564e5-22fa-4985-b988-eac4d188b3fa): CREATE TEMPORARY FUNCTION ptyUnprotectHiveDecimal as 'com.protegrity.hive.udf.ptyUnprotectHiveDecimal' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_b3d564e5-22fa-4985-b988-eac4d188b3fa); Time taken: 0.001 seconds INFO : OK No rows affected (0.046 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyProtectDate AS 'com.protegrity.hive.udf.ptyProtectDate'; INFO : Compiling command(queryId=hive_20250916121742_05f34839-7ab3-47c9-bf7c-de32b44b1305): CREATE TEMPORARY FUNCTION ptyProtectDate AS 'com.protegrity.hive.udf.ptyProtectDate' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_05f34839-7ab3-47c9-bf7c-de32b44b1305); Time taken: 0.02 seconds INFO : Executing command(queryId=hive_20250916121742_05f34839-7ab3-47c9-bf7c-de32b44b1305): CREATE TEMPORARY FUNCTION ptyProtectDate AS 'com.protegrity.hive.udf.ptyProtectDate' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_05f34839-7ab3-47c9-bf7c-de32b44b1305); Time taken: 0.001 seconds INFO : OK No rows affected (0.046 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyUnprotectDate AS 'com.protegrity.hive.udf.ptyUnprotectDate'; INFO : Compiling command(queryId=hive_20250916121742_71ac75b0-da95-409f-a070-5e06b4451973): CREATE TEMPORARY FUNCTION ptyUnprotectDate AS 'com.protegrity.hive.udf.ptyUnprotectDate' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_71ac75b0-da95-409f-a070-5e06b4451973); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121742_71ac75b0-da95-409f-a070-5e06b4451973): CREATE TEMPORARY FUNCTION ptyUnprotectDate AS 'com.protegrity.hive.udf.ptyUnprotectDate' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_71ac75b0-da95-409f-a070-5e06b4451973); Time taken: 0.001 seconds INFO : OK No rows affected (0.046 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyProtectDateTime AS 'com.protegrity.hive.udf.ptyProtectDateTime'; INFO : Compiling command(queryId=hive_20250916121742_96dfb779-c77b-4c8f-a5c0-6c3f99e5aa9c): CREATE TEMPORARY FUNCTION ptyProtectDateTime AS 'com.protegrity.hive.udf.ptyProtectDateTime' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121742_96dfb779-c77b-4c8f-a5c0-6c3f99e5aa9c); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121742_96dfb779-c77b-4c8f-a5c0-6c3f99e5aa9c): CREATE TEMPORARY FUNCTION ptyProtectDateTime AS 'com.protegrity.hive.udf.ptyProtectDateTime' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121742_96dfb779-c77b-4c8f-a5c0-6c3f99e5aa9c); Time taken: 0.001 seconds INFO : OK No rows affected (0.044 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyUnprotectDateTime AS 'com.protegrity.hive.udf.ptyUnprotectDateTime'; INFO : Compiling command(queryId=hive_20250916121743_08af3475-e39c-4cf6-9762-77c42ca6efdf): CREATE TEMPORARY FUNCTION ptyUnprotectDateTime AS 'com.protegrity.hive.udf.ptyUnprotectDateTime' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121743_08af3475-e39c-4cf6-9762-77c42ca6efdf); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121743_08af3475-e39c-4cf6-9762-77c42ca6efdf): CREATE TEMPORARY FUNCTION ptyUnprotectDateTime AS 'com.protegrity.hive.udf.ptyUnprotectDateTime' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121743_08af3475-e39c-4cf6-9762-77c42ca6efdf); Time taken: 0.001 seconds INFO : OK No rows affected (0.047 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyProtectChar AS 'com.protegrity.hive.udf.ptyProtectChar'; INFO : Compiling command(queryId=hive_20250916121743_f2ce3155-6e16-41a2-a072-d9f59142b0ea): CREATE TEMPORARY FUNCTION ptyProtectChar AS 'com.protegrity.hive.udf.ptyProtectChar' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121743_f2ce3155-6e16-41a2-a072-d9f59142b0ea); Time taken: 0.02 seconds INFO : Executing command(queryId=hive_20250916121743_f2ce3155-6e16-41a2-a072-d9f59142b0ea): CREATE TEMPORARY FUNCTION ptyProtectChar AS 'com.protegrity.hive.udf.ptyProtectChar' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121743_f2ce3155-6e16-41a2-a072-d9f59142b0ea); Time taken: 0.001 seconds INFO : OK No rows affected (0.046 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyUnprotectChar AS 'com.protegrity.hive.udf.ptyUnprotectChar'; INFO : Compiling command(queryId=hive_20250916121743_851371db-2ce1-4af0-93a0-11e83ace1b87): CREATE TEMPORARY FUNCTION ptyUnprotectChar AS 'com.protegrity.hive.udf.ptyUnprotectChar' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121743_851371db-2ce1-4af0-93a0-11e83ace1b87); Time taken: 0.02 seconds INFO : Executing command(queryId=hive_20250916121743_851371db-2ce1-4af0-93a0-11e83ace1b87): CREATE TEMPORARY FUNCTION ptyUnprotectChar AS 'com.protegrity.hive.udf.ptyUnprotectChar' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121743_851371db-2ce1-4af0-93a0-11e83ace1b87); Time taken: 0.001 seconds INFO : OK No rows affected (0.047 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyStringEnc as 'com.protegrity.hive.udf.ptyStringEnc'; INFO : Compiling command(queryId=hive_20250916121743_f72cb4f2-b99f-43c8-8a3e-a8ab7370fe94): CREATE TEMPORARY FUNCTION ptyStringEnc as 'com.protegrity.hive.udf.ptyStringEnc' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121743_f72cb4f2-b99f-43c8-8a3e-a8ab7370fe94); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121743_f72cb4f2-b99f-43c8-8a3e-a8ab7370fe94): CREATE TEMPORARY FUNCTION ptyStringEnc as 'com.protegrity.hive.udf.ptyStringEnc' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121743_f72cb4f2-b99f-43c8-8a3e-a8ab7370fe94); Time taken: 0.001 seconds INFO : OK No rows affected (0.046 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyStringDec as 'com.protegrity.hive.udf.ptyStringDec'; INFO : Compiling command(queryId=hive_20250916121743_0cc281bd-6d0c-4937-aaae-5a94f6010197): CREATE TEMPORARY FUNCTION ptyStringDec as 'com.protegrity.hive.udf.ptyStringDec' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121743_0cc281bd-6d0c-4937-aaae-5a94f6010197); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121743_0cc281bd-6d0c-4937-aaae-5a94f6010197): CREATE TEMPORARY FUNCTION ptyStringDec as 'com.protegrity.hive.udf.ptyStringDec' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121743_0cc281bd-6d0c-4937-aaae-5a94f6010197); Time taken: 0.001 seconds INFO : OK No rows affected (0.046 seconds) 0: jdbc:hive2://<master_node_name>.> CREATE TEMPORARY FUNCTION ptyStringReEnc as 'com.protegrity.hive.udf.ptyStringReEnc'; INFO : Compiling command(queryId=hive_20250916121743_ad7d4c07-9973-4f61-9afc-20df0ea9b34a): CREATE TEMPORARY FUNCTION ptyStringReEnc as 'com.protegrity.hive.udf.ptyStringReEnc' INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916121743_ad7d4c07-9973-4f61-9afc-20df0ea9b34a); Time taken: 0.019 seconds INFO : Executing command(queryId=hive_20250916121743_ad7d4c07-9973-4f61-9afc-20df0ea9b34a): CREATE TEMPORARY FUNCTION ptyStringReEnc as 'com.protegrity.hive.udf.ptyStringReEnc' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916121743_ad7d4c07-9973-4f61-9afc-20df0ea9b34a); Time taken: 0.001 seconds INFO : OK No rows affected (0.046 seconds)
Registering the Permanent Hive UDFs
Log in to the master node with a user account having permissions to create and drop UDFs.
To navigate to the directory that contains the helper script, run the following command:
cd /opt/cloudera/parcels/PTY_BDP/pephive/scriptsTo create the UDFs using the helper script, run the following command:
beeline -f create_perm_hive_udfs.hql;Execute the command in beeline after establishing a connection.
Press ENTER.
The script creates all the permanent user-defined functions for Hive.
Connected to: Apache Hive (version 3.1.3000.7.3.1.400-100) Driver: Hive JDBC (version 3.1.3000.7.3.1.400-100) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyGetVersion AS 'com.protegrity.hive.udf.ptyGetVersion'; INFO : Compiling command(queryId=hive_20250916112109_e285254e-4d3f-4485-bc15-8c97cb1fd704): CREATE FUNCTION ptyGetVersion AS 'com.protegrity.hive.udf.ptyGetVersion' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112109_e285254e-4d3f-4485-bc15-8c97cb1fd704); Time taken: 0.035 seconds INFO : Executing command(queryId=hive_20250916112109_e285254e-4d3f-4485-bc15-8c97cb1fd704): CREATE FUNCTION ptyGetVersion AS 'com.protegrity.hive.udf.ptyGetVersion' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112109_e285254e-4d3f-4485-bc15-8c97cb1fd704); Time taken: 0.019 seconds INFO : OK No rows affected (0.128 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyGetVersionExtended AS 'com.protegrity.hive.udf.ptyGetVersionExtended'; INFO : Compiling command(queryId=hive_20250916112109_318b5667-be58-48fc-a945-c44f4f3e6ce3): CREATE FUNCTION ptyGetVersionExtended AS 'com.protegrity.hive.udf.ptyGetVersionExtended' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112109_318b5667-be58-48fc-a945-c44f4f3e6ce3); Time taken: 0.028 seconds INFO : Executing command(queryId=hive_20250916112109_318b5667-be58-48fc-a945-c44f4f3e6ce3): CREATE FUNCTION ptyGetVersionExtended AS 'com.protegrity.hive.udf.ptyGetVersionExtended' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112109_318b5667-be58-48fc-a945-c44f4f3e6ce3); Time taken: 0.019 seconds INFO : OK No rows affected (0.091 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyWhoAmI AS 'com.protegrity.hive.udf.ptyWhoAmI'; INFO : Compiling command(queryId=hive_20250916112109_6f93df62-3990-42a9-9e85-aa4faf03a587): CREATE FUNCTION ptyWhoAmI AS 'com.protegrity.hive.udf.ptyWhoAmI' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112109_6f93df62-3990-42a9-9e85-aa4faf03a587); Time taken: 0.028 seconds INFO : Executing command(queryId=hive_20250916112109_6f93df62-3990-42a9-9e85-aa4faf03a587): CREATE FUNCTION ptyWhoAmI AS 'com.protegrity.hive.udf.ptyWhoAmI' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112109_6f93df62-3990-42a9-9e85-aa4faf03a587); Time taken: 0.024 seconds INFO : OK No rows affected (0.096 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyProtectStr AS 'com.protegrity.hive.udf.ptyProtectStr'; INFO : Compiling command(queryId=hive_20250916112109_ec8d621c-be21-4a7a-88e4-b7ee9bf0c23f): CREATE FUNCTION ptyProtectStr AS 'com.protegrity.hive.udf.ptyProtectStr' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112109_ec8d621c-be21-4a7a-88e4-b7ee9bf0c23f); Time taken: 0.029 seconds INFO : Executing command(queryId=hive_20250916112109_ec8d621c-be21-4a7a-88e4-b7ee9bf0c23f): CREATE FUNCTION ptyProtectStr AS 'com.protegrity.hive.udf.ptyProtectStr' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112109_ec8d621c-be21-4a7a-88e4-b7ee9bf0c23f); Time taken: 0.018 seconds INFO : OK No rows affected (0.094 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyUnprotectStr AS 'com.protegrity.hive.udf.ptyUnprotectStr'; INFO : Compiling command(queryId=hive_20250916112109_53e5e6e4-7253-4451-b88e-61fa84ea3a47): CREATE FUNCTION ptyUnprotectStr AS 'com.protegrity.hive.udf.ptyUnprotectStr' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112109_53e5e6e4-7253-4451-b88e-61fa84ea3a47); Time taken: 0.027 seconds INFO : Executing command(queryId=hive_20250916112109_53e5e6e4-7253-4451-b88e-61fa84ea3a47): CREATE FUNCTION ptyUnprotectStr AS 'com.protegrity.hive.udf.ptyUnprotectStr' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112109_53e5e6e4-7253-4451-b88e-61fa84ea3a47); Time taken: 0.017 seconds INFO : OK No rows affected (0.091 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyReprotect AS 'com.protegrity.hive.udf.ptyReprotect'; INFO : Compiling command(queryId=hive_20250916112109_294034e4-947e-4ea9-8fa3-3f6e96227c98): CREATE FUNCTION ptyReprotect AS 'com.protegrity.hive.udf.ptyReprotect' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112109_294034e4-947e-4ea9-8fa3-3f6e96227c98); Time taken: 0.03 seconds INFO : Executing command(queryId=hive_20250916112109_294034e4-947e-4ea9-8fa3-3f6e96227c98): CREATE FUNCTION ptyReprotect AS 'com.protegrity.hive.udf.ptyReprotect' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112109_294034e4-947e-4ea9-8fa3-3f6e96227c98); Time taken: 0.017 seconds INFO : OK No rows affected (0.095 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyProtectUnicode AS 'com.protegrity.hive.udf.ptyProtectUnicode'; INFO : Compiling command(queryId=hive_20250916112109_b79fdf6b-c51f-4956-ba9c-0cbf50b96060): CREATE FUNCTION ptyProtectUnicode AS 'com.protegrity.hive.udf.ptyProtectUnicode' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112109_b79fdf6b-c51f-4956-ba9c-0cbf50b96060); Time taken: 0.028 seconds INFO : Executing command(queryId=hive_20250916112109_b79fdf6b-c51f-4956-ba9c-0cbf50b96060): CREATE FUNCTION ptyProtectUnicode AS 'com.protegrity.hive.udf.ptyProtectUnicode' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112109_b79fdf6b-c51f-4956-ba9c-0cbf50b96060); Time taken: 0.024 seconds INFO : OK No rows affected (0.098 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyUnprotectUnicode AS 'com.protegrity.hive.udf.ptyUnprotectUnicode'; INFO : Compiling command(queryId=hive_20250916112110_ead2098b-374f-42ac-822d-285c50cb865d): CREATE FUNCTION ptyUnprotectUnicode AS 'com.protegrity.hive.udf.ptyUnprotectUnicode' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112110_ead2098b-374f-42ac-822d-285c50cb865d); Time taken: 0.03 seconds INFO : Executing command(queryId=hive_20250916112110_ead2098b-374f-42ac-822d-285c50cb865d): CREATE FUNCTION ptyUnprotectUnicode AS 'com.protegrity.hive.udf.ptyUnprotectUnicode' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112110_ead2098b-374f-42ac-822d-285c50cb865d); Time taken: 0.021 seconds INFO : OK No rows affected (0.112 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyReprotectUnicode AS 'com.protegrity.hive.udf.ptyReprotectUnicode'; INFO : Compiling command(queryId=hive_20250916112110_4e3502b6-cbc4-4089-9420-cef27520de02): CREATE FUNCTION ptyReprotectUnicode AS 'com.protegrity.hive.udf.ptyReprotectUnicode' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112110_4e3502b6-cbc4-4089-9420-cef27520de02); Time taken: 0.027 seconds INFO : Executing command(queryId=hive_20250916112110_4e3502b6-cbc4-4089-9420-cef27520de02): CREATE FUNCTION ptyReprotectUnicode AS 'com.protegrity.hive.udf.ptyReprotectUnicode' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112110_4e3502b6-cbc4-4089-9420-cef27520de02); Time taken: 0.017 seconds INFO : OK No rows affected (0.089 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyProtectShort AS 'com.protegrity.hive.udf.ptyProtectShort'; INFO : Compiling command(queryId=hive_20250916112110_58e9c7fa-3cda-4e52-9d49-7e048e29f6ac): CREATE FUNCTION ptyProtectShort AS 'com.protegrity.hive.udf.ptyProtectShort' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112110_58e9c7fa-3cda-4e52-9d49-7e048e29f6ac); Time taken: 0.027 seconds INFO : Executing command(queryId=hive_20250916112110_58e9c7fa-3cda-4e52-9d49-7e048e29f6ac): CREATE FUNCTION ptyProtectShort AS 'com.protegrity.hive.udf.ptyProtectShort' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112110_58e9c7fa-3cda-4e52-9d49-7e048e29f6ac); Time taken: 0.018 seconds INFO : OK No rows affected (0.089 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyUnprotectShort AS 'com.protegrity.hive.udf.ptyUnprotectShort'; INFO : Compiling command(queryId=hive_20250916112110_3c257d7e-2032-43b7-808f-c4bcd4e74435): CREATE FUNCTION ptyUnprotectShort AS 'com.protegrity.hive.udf.ptyUnprotectShort' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112110_3c257d7e-2032-43b7-808f-c4bcd4e74435); Time taken: 0.027 seconds INFO : Executing command(queryId=hive_20250916112110_3c257d7e-2032-43b7-808f-c4bcd4e74435): CREATE FUNCTION ptyUnprotectShort AS 'com.protegrity.hive.udf.ptyUnprotectShort' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112110_3c257d7e-2032-43b7-808f-c4bcd4e74435); Time taken: 0.017 seconds INFO : OK No rows affected (0.091 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyProtectInt AS 'com.protegrity.hive.udf.ptyProtectInt'; INFO : Compiling command(queryId=hive_20250916112110_368c452c-c0ab-4887-ac46-5bb5ff1b487a): CREATE FUNCTION ptyProtectInt AS 'com.protegrity.hive.udf.ptyProtectInt' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112110_368c452c-c0ab-4887-ac46-5bb5ff1b487a); Time taken: 0.027 seconds INFO : Executing command(queryId=hive_20250916112110_368c452c-c0ab-4887-ac46-5bb5ff1b487a): CREATE FUNCTION ptyProtectInt AS 'com.protegrity.hive.udf.ptyProtectInt' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112110_368c452c-c0ab-4887-ac46-5bb5ff1b487a); Time taken: 0.019 seconds INFO : OK No rows affected (0.089 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyUnprotectInt AS 'com.protegrity.hive.udf.ptyUnprotectInt'; INFO : Compiling command(queryId=hive_20250916112110_68178046-266b-4e42-834c-9fc2261c6b47): CREATE FUNCTION ptyUnprotectInt AS 'com.protegrity.hive.udf.ptyUnprotectInt' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112110_68178046-266b-4e42-834c-9fc2261c6b47); Time taken: 0.027 seconds INFO : Executing command(queryId=hive_20250916112110_68178046-266b-4e42-834c-9fc2261c6b47): CREATE FUNCTION ptyUnprotectInt AS 'com.protegrity.hive.udf.ptyUnprotectInt' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112110_68178046-266b-4e42-834c-9fc2261c6b47); Time taken: 0.024 seconds INFO : OK No rows affected (0.097 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyProtectBigInt as 'com.protegrity.hive.udf.ptyProtectBigInt'; INFO : Compiling command(queryId=hive_20250916112110_14c69433-7fcd-4b48-b5e2-a091b28dd22a): CREATE FUNCTION ptyProtectBigInt as 'com.protegrity.hive.udf.ptyProtectBigInt' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112110_14c69433-7fcd-4b48-b5e2-a091b28dd22a); Time taken: 0.027 seconds INFO : Executing command(queryId=hive_20250916112110_14c69433-7fcd-4b48-b5e2-a091b28dd22a): CREATE FUNCTION ptyProtectBigInt as 'com.protegrity.hive.udf.ptyProtectBigInt' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112110_14c69433-7fcd-4b48-b5e2-a091b28dd22a); Time taken: 0.019 seconds INFO : OK No rows affected (0.091 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyUnprotectBigInt as 'com.protegrity.hive.udf.ptyUnprotectBigInt'; INFO : Compiling command(queryId=hive_20250916112110_0a3767a6-4db2-4645-81c4-c4f874c7015c): CREATE FUNCTION ptyUnprotectBigInt as 'com.protegrity.hive.udf.ptyUnprotectBigInt' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112110_0a3767a6-4db2-4645-81c4-c4f874c7015c); Time taken: 0.028 seconds INFO : Executing command(queryId=hive_20250916112110_0a3767a6-4db2-4645-81c4-c4f874c7015c): CREATE FUNCTION ptyUnprotectBigInt as 'com.protegrity.hive.udf.ptyUnprotectBigInt' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112110_0a3767a6-4db2-4645-81c4-c4f874c7015c); Time taken: 0.02 seconds INFO : OK No rows affected (0.091 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyProtectFloat as 'com.protegrity.hive.udf.ptyProtectFloat'; INFO : Compiling command(queryId=hive_20250916112110_3f894aa3-4bb4-47f4-ac7e-c24021dbc7fb): CREATE FUNCTION ptyProtectFloat as 'com.protegrity.hive.udf.ptyProtectFloat' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112110_3f894aa3-4bb4-47f4-ac7e-c24021dbc7fb); Time taken: 0.028 seconds INFO : Executing command(queryId=hive_20250916112110_3f894aa3-4bb4-47f4-ac7e-c24021dbc7fb): CREATE FUNCTION ptyProtectFloat as 'com.protegrity.hive.udf.ptyProtectFloat' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112110_3f894aa3-4bb4-47f4-ac7e-c24021dbc7fb); Time taken: 0.019 seconds INFO : OK No rows affected (0.091 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyUnprotectFloat as 'com.protegrity.hive.udf.ptyProtectFloat'; INFO : Compiling command(queryId=hive_20250916112110_7a9ee32b-f391-4eff-a54e-53577961dd33): CREATE FUNCTION ptyUnprotectFloat as 'com.protegrity.hive.udf.ptyProtectFloat' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112110_7a9ee32b-f391-4eff-a54e-53577961dd33); Time taken: 0.028 seconds INFO : Executing command(queryId=hive_20250916112110_7a9ee32b-f391-4eff-a54e-53577961dd33): CREATE FUNCTION ptyUnprotectFloat as 'com.protegrity.hive.udf.ptyProtectFloat' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112110_7a9ee32b-f391-4eff-a54e-53577961dd33); Time taken: 0.023 seconds INFO : OK No rows affected (0.096 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyProtectDouble as 'com.protegrity.hive.udf.ptyProtectDouble'; INFO : Compiling command(queryId=hive_20250916112111_3f9d2bfd-9fda-4af6-be16-d620370ccffc): CREATE FUNCTION ptyProtectDouble as 'com.protegrity.hive.udf.ptyProtectDouble' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112111_3f9d2bfd-9fda-4af6-be16-d620370ccffc); Time taken: 0.026 seconds INFO : Executing command(queryId=hive_20250916112111_3f9d2bfd-9fda-4af6-be16-d620370ccffc): CREATE FUNCTION ptyProtectDouble as 'com.protegrity.hive.udf.ptyProtectDouble' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112111_3f9d2bfd-9fda-4af6-be16-d620370ccffc); Time taken: 0.019 seconds INFO : OK No rows affected (0.093 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyUnprotectDouble as 'com.protegrity.hive.udf.ptyUnprotectDouble'; INFO : Compiling command(queryId=hive_20250916112111_d4a442ea-3e40-4096-a760-caa65aaea9c1): CREATE FUNCTION ptyUnprotectDouble as 'com.protegrity.hive.udf.ptyUnprotectDouble' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112111_d4a442ea-3e40-4096-a760-caa65aaea9c1); Time taken: 0.028 seconds INFO : Executing command(queryId=hive_20250916112111_d4a442ea-3e40-4096-a760-caa65aaea9c1): CREATE FUNCTION ptyUnprotectDouble as 'com.protegrity.hive.udf.ptyUnprotectDouble' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112111_d4a442ea-3e40-4096-a760-caa65aaea9c1); Time taken: 0.017 seconds INFO : OK No rows affected (0.09 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyProtectDec as 'com.protegrity.hive.udf.ptyProtectDec'; INFO : Compiling command(queryId=hive_20250916112111_f024de0b-ba3f-44ef-b2d6-1faa94ee0732): CREATE FUNCTION ptyProtectDec as 'com.protegrity.hive.udf.ptyProtectDec' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112111_f024de0b-ba3f-44ef-b2d6-1faa94ee0732); Time taken: 0.027 seconds INFO : Executing command(queryId=hive_20250916112111_f024de0b-ba3f-44ef-b2d6-1faa94ee0732): CREATE FUNCTION ptyProtectDec as 'com.protegrity.hive.udf.ptyProtectDec' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112111_f024de0b-ba3f-44ef-b2d6-1faa94ee0732); Time taken: 0.017 seconds INFO : OK No rows affected (0.087 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyUnprotectDec as 'com.protegrity.hive.udf.ptyUnprotectDec'; INFO : Compiling command(queryId=hive_20250916112111_1e7c1d77-ecdf-4ab1-b6e2-e0421b34f29c): CREATE FUNCTION ptyUnprotectDec as 'com.protegrity.hive.udf.ptyUnprotectDec' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112111_1e7c1d77-ecdf-4ab1-b6e2-e0421b34f29c); Time taken: 0.027 seconds INFO : Executing command(queryId=hive_20250916112111_1e7c1d77-ecdf-4ab1-b6e2-e0421b34f29c): CREATE FUNCTION ptyUnprotectDec as 'com.protegrity.hive.udf.ptyUnprotectDec' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112111_1e7c1d77-ecdf-4ab1-b6e2-e0421b34f29c); Time taken: 0.017 seconds INFO : OK No rows affected (0.087 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyProtectHiveDecimal as 'com.protegrity.hive.udf.ptyProtectHiveDecimal'; INFO : Compiling command(queryId=hive_20250916112111_0edaca35-32d5-49d9-9a3a-d6e644de1afd): CREATE FUNCTION ptyProtectHiveDecimal as 'com.protegrity.hive.udf.ptyProtectHiveDecimal' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112111_0edaca35-32d5-49d9-9a3a-d6e644de1afd); Time taken: 0.027 seconds INFO : Executing command(queryId=hive_20250916112111_0edaca35-32d5-49d9-9a3a-d6e644de1afd): CREATE FUNCTION ptyProtectHiveDecimal as 'com.protegrity.hive.udf.ptyProtectHiveDecimal' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112111_0edaca35-32d5-49d9-9a3a-d6e644de1afd); Time taken: 0.017 seconds INFO : OK No rows affected (0.088 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyUnprotectHiveDecimal as 'com.protegrity.hive.udf.ptyUnprotectHiveDecimal'; INFO : Compiling command(queryId=hive_20250916112111_399c586f-a1eb-4d1c-b750-d704a13061b1): CREATE FUNCTION ptyUnprotectHiveDecimal as 'com.protegrity.hive.udf.ptyUnprotectHiveDecimal' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112111_399c586f-a1eb-4d1c-b750-d704a13061b1); Time taken: 0.026 seconds INFO : Executing command(queryId=hive_20250916112111_399c586f-a1eb-4d1c-b750-d704a13061b1): CREATE FUNCTION ptyUnprotectHiveDecimal as 'com.protegrity.hive.udf.ptyUnprotectHiveDecimal' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112111_399c586f-a1eb-4d1c-b750-d704a13061b1); Time taken: 0.023 seconds INFO : OK No rows affected (0.094 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyProtectDate AS 'com.protegrity.hive.udf.ptyProtectDate'; INFO : Compiling command(queryId=hive_20250916112111_0343ff3f-90a3-4b51-9276-1f53a95aae27): CREATE FUNCTION ptyProtectDate AS 'com.protegrity.hive.udf.ptyProtectDate' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112111_0343ff3f-90a3-4b51-9276-1f53a95aae27); Time taken: 0.027 seconds INFO : Executing command(queryId=hive_20250916112111_0343ff3f-90a3-4b51-9276-1f53a95aae27): CREATE FUNCTION ptyProtectDate AS 'com.protegrity.hive.udf.ptyProtectDate' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112111_0343ff3f-90a3-4b51-9276-1f53a95aae27); Time taken: 0.017 seconds INFO : OK No rows affected (0.086 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyUnprotectDate AS 'com.protegrity.hive.udf.ptyUnprotectDate'; INFO : Compiling command(queryId=hive_20250916112111_de190f84-ba1c-41e4-bdb3-e0efd091eb48): CREATE FUNCTION ptyUnprotectDate AS 'com.protegrity.hive.udf.ptyUnprotectDate' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112111_de190f84-ba1c-41e4-bdb3-e0efd091eb48); Time taken: 0.027 seconds INFO : Executing command(queryId=hive_20250916112111_de190f84-ba1c-41e4-bdb3-e0efd091eb48): CREATE FUNCTION ptyUnprotectDate AS 'com.protegrity.hive.udf.ptyUnprotectDate' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112111_de190f84-ba1c-41e4-bdb3-e0efd091eb48); Time taken: 0.018 seconds INFO : OK No rows affected (0.09 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyProtectDateTime AS 'com.protegrity.hive.udf.ptyProtectDateTime'; INFO : Compiling command(queryId=hive_20250916112111_0189db86-a9dc-4934-a15f-08d08aa238d5): CREATE FUNCTION ptyProtectDateTime AS 'com.protegrity.hive.udf.ptyProtectDateTime' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112111_0189db86-a9dc-4934-a15f-08d08aa238d5); Time taken: 0.027 seconds INFO : Executing command(queryId=hive_20250916112111_0189db86-a9dc-4934-a15f-08d08aa238d5): CREATE FUNCTION ptyProtectDateTime AS 'com.protegrity.hive.udf.ptyProtectDateTime' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112111_0189db86-a9dc-4934-a15f-08d08aa238d5); Time taken: 0.017 seconds INFO : OK No rows affected (0.088 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyUnprotectDateTime AS 'com.protegrity.hive.udf.ptyUnprotectDateTime'; INFO : Compiling command(queryId=hive_20250916112111_c629e410-5d0a-40ef-bb26-8daa6e3546bb): CREATE FUNCTION ptyUnprotectDateTime AS 'com.protegrity.hive.udf.ptyUnprotectDateTime' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112111_c629e410-5d0a-40ef-bb26-8daa6e3546bb); Time taken: 0.028 seconds INFO : Executing command(queryId=hive_20250916112111_c629e410-5d0a-40ef-bb26-8daa6e3546bb): CREATE FUNCTION ptyUnprotectDateTime AS 'com.protegrity.hive.udf.ptyUnprotectDateTime' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112111_c629e410-5d0a-40ef-bb26-8daa6e3546bb); Time taken: 0.024 seconds INFO : OK No rows affected (0.094 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyProtectChar AS 'com.protegrity.hive.udf.ptyProtectChar'; INFO : Compiling command(queryId=hive_20250916112112_be9b293c-6336-45d8-825a-480e31c54700): CREATE FUNCTION ptyProtectChar AS 'com.protegrity.hive.udf.ptyProtectChar' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112112_be9b293c-6336-45d8-825a-480e31c54700); Time taken: 0.026 seconds INFO : Executing command(queryId=hive_20250916112112_be9b293c-6336-45d8-825a-480e31c54700): CREATE FUNCTION ptyProtectChar AS 'com.protegrity.hive.udf.ptyProtectChar' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112112_be9b293c-6336-45d8-825a-480e31c54700); Time taken: 0.017 seconds INFO : OK No rows affected (0.086 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyUnprotectChar AS 'com.protegrity.hive.udf.ptyUnprotectChar'; INFO : Compiling command(queryId=hive_20250916112112_386a3434-a990-4180-a23a-f684fcfe391c): CREATE FUNCTION ptyUnprotectChar AS 'com.protegrity.hive.udf.ptyUnprotectChar' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112112_386a3434-a990-4180-a23a-f684fcfe391c); Time taken: 0.026 seconds INFO : Executing command(queryId=hive_20250916112112_386a3434-a990-4180-a23a-f684fcfe391c): CREATE FUNCTION ptyUnprotectChar AS 'com.protegrity.hive.udf.ptyUnprotectChar' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112112_386a3434-a990-4180-a23a-f684fcfe391c); Time taken: 0.016 seconds INFO : OK No rows affected (0.086 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyStringEnc as 'com.protegrity.hive.udf.ptyStringEnc'; INFO : Compiling command(queryId=hive_20250916112112_7f32e8f6-4688-41b8-b025-b6eb9ddb45fd): CREATE FUNCTION ptyStringEnc as 'com.protegrity.hive.udf.ptyStringEnc' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112112_7f32e8f6-4688-41b8-b025-b6eb9ddb45fd); Time taken: 0.026 seconds INFO : Executing command(queryId=hive_20250916112112_7f32e8f6-4688-41b8-b025-b6eb9ddb45fd): CREATE FUNCTION ptyStringEnc as 'com.protegrity.hive.udf.ptyStringEnc' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112112_7f32e8f6-4688-41b8-b025-b6eb9ddb45fd); Time taken: 0.017 seconds INFO : OK No rows affected (0.087 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyStringDec as 'com.protegrity.hive.udf.ptyStringDec'; INFO : Compiling command(queryId=hive_20250916112112_c2be0c4b-5208-4799-bcc3-f591ae30718a): CREATE FUNCTION ptyStringDec as 'com.protegrity.hive.udf.ptyStringDec' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112112_c2be0c4b-5208-4799-bcc3-f591ae30718a); Time taken: 0.027 seconds INFO : Executing command(queryId=hive_20250916112112_c2be0c4b-5208-4799-bcc3-f591ae30718a): CREATE FUNCTION ptyStringDec as 'com.protegrity.hive.udf.ptyStringDec' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112112_c2be0c4b-5208-4799-bcc3-f591ae30718a); Time taken: 0.017 seconds INFO : OK No rows affected (0.085 seconds) 0: jdbc:hive2://<master_node_name>> CREATE FUNCTION ptyStringReEnc as 'com.protegrity.hive.udf.ptyStringReEnc'; INFO : Compiling command(queryId=hive_20250916112112_99ffe441-5bf8-4ab9-9746-78d43e1306eb): CREATE FUNCTION ptyStringReEnc as 'com.protegrity.hive.udf.ptyStringReEnc' WARN : permanent functions created without USING clause will not be replicated. INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20250916112112_99ffe441-5bf8-4ab9-9746-78d43e1306eb); Time taken: 0.027 seconds INFO : Executing command(queryId=hive_20250916112112_99ffe441-5bf8-4ab9-9746-78d43e1306eb): CREATE FUNCTION ptyStringReEnc as 'com.protegrity.hive.udf.ptyStringReEnc' INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20250916112112_99ffe441-5bf8-4ab9-9746-78d43e1306eb); Time taken: 0.016 seconds INFO : OK No rows affected (0.088 seconds)
Registering the Impala user-defined functions
Navigate to the S3 bucket where the installation files are uploaded.
Open the createobjects.sql file.
Copy the contents of the createobjects.sql file.
Log in to the master node with a user account having permissions to create and drop UDFs.
To create the UDFs using the helper script, run the following command:
impala-shell -i <IP_Address_of_node> -kPress ENTER.
The script creates all the required user-defined functions for Impala.
CREATE FUNCTION pty_getversion() RETURNS STRING > LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so' > SYMBOL = 'pty_getversion';
SYMBOL = ‘pty_getversionextended’;
Query: CREATE FUNCTION pty_getversion() RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_getversion'
CREATE FUNCTION pty_whoami() RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_whoami';
CREATE FUNCTION pty_stringenc(STRING, STRING) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_stringenc' prepare_fn='UdfPrepare' close_fn='UdfClose';
CREATE FUNCTION pty_stringdec(STRING, STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_stringdec' prepare_fn='UdfPrepare' close_fn='UdfClose';
CREATE FUNCTION pty_stringins(STRING,STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_stringins' prepare_fn='UdfPrepare' close_fn='UdfClose';
CREATE FUNCTION pty_stringsel(STRING, STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_stringsel' prepare_fn='UdfPrepare' close_fn='UdfClose';
CREATE FUNCTION pty_unicodestringins(STRING,STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_unicodestringins' prepare_fn='UdfPrepare' close_fn='UdfClose';
CREATE FUNCTION pty_unicodestringsel(STRING,STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_unicodestringsel' prepare_fn='UdfPrepare' close_fn='UdfClose';
CREATE FUNCTION pty_unicodestringfpeins(STRING,STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_unicodestringfpeins' prepare_fn='UdfPrepare' close_fn='UdfClose';
CREATE FUNCTION pty_unicodestringfpesel(STRING,STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_unicodestringfpesel' prepare_fn='UdfPrepare' close_fn='UdfClose';
CREATE FUNCTION pty_integerenc(INTEGER, STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_integerenc' prepare_fn='UdfPrepare' close_fn='UdfClose';
CREATE FUNCTION pty_integerdec(STRING, STRING ) RETURNS INTEGER
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_integerdec' prepare_fn='UdfPrepare' close_fn='UdfClose';
CREATE FUNCTION pty_integerins(INTEGER, STRING ) RETURNS INTEGER
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_integerins' prepare_fn='UdfPrepare' close_fn='UdfClose';
CREATE FUNCTION pty_integersel(INTEGER, STRING ) RETURNS INTEGER
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_integersel' prepare_fn='UdfPrepare' close_fn='UdfClose';
CREATE FUNCTION pty_doubleenc(double, STRING ) RETURNS string
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_doubleenc' prepare_fn='UdfPrepare' close_fn='UdfClose';
CREATE FUNCTION pty_doubledec(STRING, STRING ) RETURNS double
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_doubledec' prepare_fn='UdfPrepare' close_fn='UdfClose';
CREATE FUNCTION pty_doubleins(double, STRING ) RETURNS double
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_doubleins' prepare_fn='UdfPrepare' close_fn='UdfClose';
CREATE FUNCTION pty_doublesel(DOUBLE, STRING ) RETURNS DOUBL+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.67s
default>
>
> CREATE FUNCTION pty_getversionextended() RETURNS STRING
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_getversionextended';
Query: CREATE FUNCTION pty_getversionextended() RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_getversionextended'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.21s
default>
f default>
l default> CREATE FUNCTION pty_whoami() RETURNS STRING
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_whoami';
Query: CREATE FUNCTION pty_whoami() RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_whoami'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.20s
default>
>
> CREATE FUNCTION pty_stringenc(STRING, STRING) RETURNS STRING
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_stringenc' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_stringenc(STRING, STRING) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_stringenc' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.25s
default>
>
d > CREATE FUNCTION pty_stringdec(STRING, STRING ) RETURNS STRING
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_stringdec' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_stringdec(STRING, STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_stringdec' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.25s
default>
default> CREATE FUNCTION pty_stringins(STRING,STRING ) RETURNS STRING
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_stringins' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_stringins(STRING,STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_stringins' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.24s
default>
default> CREATE FUNCTION pty_stringsel(STRING, STRING ) RETURNS STRING
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_stringsel' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_stringsel(STRING, STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_stringsel' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.28s
default>
U default> CREATE FUNCTION pty_unicodestringins(STRING,STRING ) RETURNS STRING
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_unicodestringins' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_unicodestringins(STRING,STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_unicodestringins' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.25s
default>
default> CREATE FUNCTION pty_unicodestringsel(STRING,STRING ) RETURNS STRING
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_unicodestringsel' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_unicodestringsel(STRING,STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_unicodestringsel' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.24s
default>
default> CREATE FUNCTION pty_unicodestringfpeins(STRING,STRING ) RETURNS STRING
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_unicodestringfpeins' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_unicodestringfpeins(STRING,STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_unicodestringfpeins' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.24s
default>
default> CREATE FUNCTION pty_unicodestringfpesel(STRING,STRING ) RETURNS STRING
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_unicodestringfpesel' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_unicodestringfpesel(STRING,STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_unicodestringfpesel' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.27s
default>
default> CREATE FUNCTION pty_integerenc(INTEGER, STRING ) RETURNS STRING
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_integerenc' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_integerenc(INTEGER, STRING ) RETURNS STRING
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_integerenc' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.27s
default>
I default> CREATE FUNCTION pty_integerdec(STRING, STRING ) RETURNS INTEGER
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_integerdec' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_integerdec(STRING, STRING ) RETURNS INTEGER
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_integerdec' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.26s
default>
t default> CREATE FUNCTION pty_integerins(INTEGER, STRING ) RETURNS INTEGER
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_integerins' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_integerins(INTEGER, STRING ) RETURNS INTEGER
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_integerins' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.26s
default>
e default> CREATE FUNCTION pty_integersel(INTEGER, STRING ) RETURNS INTEGER
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_integersel' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_integersel(INTEGER, STRING ) RETURNS INTEGER
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_integersel' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.28s
default>
, default> CREATE FUNCTION pty_doubleenc(double, STRING ) RETURNS string
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_doubleenc' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_doubleenc(double, STRING ) RETURNS string
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_doubleenc' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.26s
default>
default> CREATE FUNCTION pty_doubledec(STRING, STRING ) RETURNS double
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_doubledec' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_doubledec(STRING, STRING ) RETURNS double
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_doubledec' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.25s
default>
default> CREATE FUNCTION pty_doubleins(double, STRING ) RETURNS double
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_doubleins' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_doubleins(double, STRING ) RETURNS double
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_doubleins' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.28s
default>
default> CREATE FUNCTION pty_doublesel(DOUBLE, STRING ) RETURNS DOUBLE
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_doublesel' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_doublesel(DOUBLE, STRING ) RETURNS DOUBLE
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_doublesel' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.23s
default>
default> CREATE FUNCTION pty_floatenc(float, STRING ) RETURNS string
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_floatenc' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_floatenc(float, STRING ) RETURNS string
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_floatenc' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.27s
default>
default> CREATE FUNCTION pty_floatdec(STRING, STRING ) RETURNS float
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_floatdec' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_floatdec(STRING, STRING ) RETURNS float
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_floatdec' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.24s
default>
default> CREATE FUNCTION pty_floatins(float, STRING ) RETURNS float
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_floatins' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_floatins(float, STRING ) RETURNS float
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_floatins' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.29s
default>
default> CREATE FUNCTION pty_floatsel(float, STRING ) RETURNS float
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_floatsel' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_floatsel(float, STRING ) RETURNS float
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_floatsel' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.23s
default>
default> CREATE FUNCTION pty_smallintenc(smallint, STRING ) RETURNS string
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_smallintenc' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_smallintenc(smallint, STRING ) RETURNS string
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_smallintenc' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.24s
default>
default> CREATE FUNCTION pty_smallintdec(STRING, STRING ) RETURNS smallint
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_smallintdec' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_smallintdec(STRING, STRING ) RETURNS smallint
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_smallintdec' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.28s
default>
default> CREATE FUNCTION pty_smallintins(smallint, STRING ) RETURNS smallint
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_smallintins' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_smallintins(smallint, STRING ) RETURNS smallint
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_smallintins' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.25s
default>
default> CREATE FUNCTION pty_smallintsel(smallint, STRING ) RETURNS smallint
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_smallintsel' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_smallintsel(smallint, STRING ) RETURNS smallint
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_smallintsel' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.24s
default>
default> CREATE FUNCTION pty_bigintenc(bigint, STRING) RETURNS string
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_bigintenc' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_bigintenc(bigint, STRING) RETURNS string
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_bigintenc' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.24s
default>
default> CREATE FUNCTION pty_bigintdec(STRING, STRING) RETURNS bigint
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_bigintdec' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_bigintdec(STRING, STRING) RETURNS bigint
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_bigintdec' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.25s
default>
default> CREATE FUNCTION pty_bigintins(bigint, STRING) RETURNS bigint
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_bigintins' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_bigintins(bigint, STRING) RETURNS bigint
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_bigintins' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.24s
default>
default> CREATE FUNCTION pty_bigintsel(bigint, STRING) RETURNS bigint
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_bigintsel' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_bigintsel(bigint, STRING) RETURNS bigint
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_bigintsel' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.24s
default>
default> CREATE FUNCTION pty_dateenc(date, STRING ) RETURNS string
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_dateenc' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_dateenc(date, STRING ) RETURNS string
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_dateenc' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.27s
default>
default> CREATE FUNCTION pty_datedec(STRING, STRING ) RETURNS date
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_datedec' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_datedec(STRING, STRING ) RETURNS date
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_datedec' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.24s
default>
default> CREATE FUNCTION pty_dateins(date, STRING ) RETURNS date
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_dateins' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_dateins(date, STRING ) RETURNS date
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_dateins' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.22s
default>
default> CREATE FUNCTION pty_datesel(date, STRING ) RETURNS date
> LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
> SYMBOL = 'pty_datesel' prepare_fn='UdfPrepare' close_fn='UdfClose';
Query: CREATE FUNCTION pty_datesel(date, STRING ) RETURNS date
LOCATION 's3a://<bucket_name>/<directory_name>/pepimpala/pepimpala4_0_RHEL.so'
SYMBOL = 'pty_datesel' prepare_fn='UdfPrepare' close_fn='UdfClose'
+----------------------------+
| summary |
+----------------------------+
| Function has been created. |
+----------------------------+
Fetched 1 row(s) in 0.24s
```
Registering the SparkSQL user-defined functions
Log in to the master node with a user account having permissions to create and drop UDFs.
To navigate to the directory that contains the helper script, run the following command:
cd /opt/cloudera/parcels/PTY_BDP/pepspark/scriptsTo create the UDFs using the helper script, run the following command in the spark-shell:
:load /opt/cloudera/parcels/PTY_BDP/pepspark/scripts/create_spark_sql_udfs.scalaPress ENTER.
The script creates all the required user-defined functions for SparkSQL.
Loading /opt/cloudera/parcels/PTY_BDP/pepspark/scripts/create_spark_sql_udfs.scala... res0: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2557/1214243533@e9f28,StringType,List(),Some(class[value[0]: string]),Some(ptyGetVersion),true,true) res1: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2603/321785376@684ad81c,StringType,List(),Some(class[value[0]: string]),Some(ptyGetVersionExtended),true,true) res2: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2604/289080194@594bedf5,StringType,List(),Some(class[value[0]: string]),Some(ptyWhoAmI),true,true) res3: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2605/430442099@6ec6adcc,StringType,List(Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: string]),Some(ptyProtectStr),true,true) res4: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2612/1566019818@55b678dc,StringType,List(Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: string]),Some(ptyUnprotectStr),true,true) res5: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2613/1992744664@2dff4ef9,StringType,List(Some(class[value[0]: string]), Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: string]),Some(ptyReprotectStr),true,true) res6: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2621/2144907913@4d13970d,StringType,List(Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: string]),Some(ptyProtectUnicode),true,true) res7: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2622/567181258@7c8d4a94,StringType,List(Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: string]),Some(ptyUnprotectUnicode),true,true) res8: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2623/1248911890@590eb2c5,StringType,List(Some(class[value[0]: string]), Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: string]),Some(ptyReprotectUnicode),true,true) res9: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2639/1206966491@4e3617fe,ShortType,List(Some(class[value[0]: smallint]), Some(class[value[0]: string])),Some(class[value[0]: smallint]),Some(ptyProtectShort),false,true) res10: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2643/1430577369@5056f8d7,ShortType,List(Some(class[value[0]: smallint]), Some(class[value[0]: string])),Some(class[value[0]: smallint]),Some(ptyUnprotectShort),false,true) res11: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2644/1959246940@3e7d458a,ShortType,List(Some(class[value[0]: smallint]), Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: smallint]),Some(ptyReprotectShort),false,true) res12: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2646/468430240@6b874125,IntegerType,List(Some(class[value[0]: int]), Some(class[value[0]: string])),Some(class[value[0]: int]),Some(ptyProtectInt),false,true) res13: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2648/1849024377@377b8c99,IntegerType,List(Some(class[value[0]: int]), Some(class[value[0]: string])),Some(class[value[0]: int]),Some(ptyUnprotectInt),false,true) res14: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2649/1850050643@1ddbf1b0,IntegerType,List(Some(class[value[0]: int]), Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: int]),Some(ptyReprotectInt),false,true) res15: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2650/1751709974@65f23702,LongType,List(Some(class[value[0]: bigint]), Some(class[value[0]: string])),Some(class[value[0]: bigint]),Some(ptyProtectLong),false,true) res16: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2652/1397163963@5d98ac30,LongType,List(Some(class[value[0]: bigint]), Some(class[value[0]: string])),Some(class[value[0]: bigint]),Some(ptyUnprotectLong),false,true) res17: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2653/231449448@5ce648c7,LongType,List(Some(class[value[0]: bigint]), Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: bigint]),Some(ptyReprotectLong),false,true) res18: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2654/916221467@203dff48,FloatType,List(Some(class[value[0]: float]), Some(class[value[0]: string])),Some(class[value[0]: float]),Some(ptyProtectFloat),false,true) res19: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2656/1642716671@2403ecd0,FloatType,List(Some(class[value[0]: float]), Some(class[value[0]: string])),Some(class[value[0]: float]),Some(ptyUnprotectFloat),false,true) res20: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2657/449484397@780f6346,FloatType,List(Some(class[value[0]: float]), Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: float]),Some(ptyReprotectFloat),false,true) res21: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2658/311232024@4718da4b,DoubleType,List(Some(class[value[0]: double]), Some(class[value[0]: string])),Some(class[value[0]: double]),Some(ptyProtectDouble),false,true) res22: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2660/1882823613@136e7e2c,DoubleType,List(Some(class[value[0]: double]), Some(class[value[0]: string])),Some(class[value[0]: double]),Some(ptyUnprotectDouble),false,true) res23: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2661/1574577816@2f4f900d,DoubleType,List(Some(class[value[0]: double]), Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: double]),Some(ptyReprotectDouble),false,true) res24: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2662/701508258@404d6f2,DateType,List(Some(class[value[0]: date]), Some(class[value[0]: string])),Some(class[value[0]: date]),Some(ptyProtectDate),true,true) res25: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2673/1441934479@512f3e71,DateType,List(Some(class[value[0]: date]), Some(class[value[0]: string])),Some(class[value[0]: date]),Some(ptyUnprotectDate),true,true) res26: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2674/19354823@7bacb1b0,DateType,List(Some(class[value[0]: date]), Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: date]),Some(ptyReprotectDate),true,true) res27: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2675/1203531300@31fe39d3,TimestampType,List(Some(class[value[0]: timestamp]), Some(class[value[0]: string])),Some(class[value[0]: timestamp]),Some(ptyProtectDateTime),true,true) res28: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2676/1395761147@5d81b1ef,TimestampType,List(Some(class[value[0]: timestamp]), Some(class[value[0]: string])),Some(class[value[0]: timestamp]),Some(ptyUnprotectDateTime),true,true) res29: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2677/971152222@1af59a5e,TimestampType,List(Some(class[value[0]: timestamp]), Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: timestamp]),Some(ptyReprotectDateTime),true,true) res30: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2678/449445798@4f994c53,DecimalType(38,18),List(Some(class[value[0]: decimal(38,18)]), Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: decimal(38,18)]),Some(ptyProtectDecimal),true,true) res31: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2687/375594857@7f5ae905,DecimalType(38,18),List(Some(class[value[0]: decimal(38,18)]), Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: decimal(38,18)]),Some(ptyUnprotectDecimal),true,true) res32: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2688/2133807474@33f1f5a,DecimalType(38,18),List(Some(class[value[0]: decimal(38,18)]), Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: decimal(38,18)]),Some(ptyReprotectDecimal),true,true) res33: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2691/1933809761@d57894d,BinaryType,List(Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: binary]),Some(ptyStringEnc),true,true) res34: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2693/255369243@25ed9699,StringType,List(Some(class[value[0]: binary]), Some(class[value[0]: string])),Some(class[value[0]: string]),Some(ptyStringDec),true,true) res35: org.apache.spark.sql.expressions.UserDefinedFunction = SparkUserDefinedFunction($Lambda$2694/542980564@7382cd26,BinaryType,List(Some(class[value[0]: binary]), Some(class[value[0]: string]), Some(class[value[0]: string])),Some(class[value[0]: binary]),Some(ptyStringReEnc),true,true)
Registering the PySpark Scala Wrapper user-defined functions
Log in to the master node with a user account having permissions to create and drop UDFs.
To navigate to the directory that contains the helper script, run the following command:
cd /opt/cloudera/parcels/PTY_BDP/pepspark/scriptsTo create the UDFs using the helper script, run the following command in the
pysparkshell:exec(open("/opt/cloudera/parcels/PTY_BDP/pepspark/scripts/create_scala_wrapper_udfs.py").read());Press ENTER.
The script creates all the required Scala Wrapper user-defined functions.
Feedback
Was this page helpful?