This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Protegrity Anonymization

Using Protegrity Anonymization with PPC

Protegrity Anonymization is a software solution that removes personal information and transforms data attributes to maintain privacy. It takes raw data as input, applies techniques such as generalization and summarization, and outputs anonymized data. You can use this output for analysis without revealing individual identities.

For more information about Protegrity Anonymization, refer to Protegrity Anonymization.

1 - Prerequisites

List of Prerequisites for Protegrity Anonymization.

Ensure the following prerequisites are met:

Tools

Install and configure the following tools on the jumpbox:

  • opentofu 1.10 or later. For more information about installation steps, refer to OpenTofu Installation.
  • helm 3.12+, kubectl 1.27+, and aws CLI v2 with access to the Protegrity Provisioned Cluster (PPC).

AWS Setup

Ensure that the following AWS requirements are met:

  • A Protegrity Provisioned Cluster (PPC) is available. For more information about PPC, refer to Protegrity Provisioned Cluster.

  • AWS CLI credentials are configured using aws configure.

  • The AWS credentials have permission to provision IAM roles, S3 buckets, EKS Pod Identity associations, and Karpenter node resources. If you encounter AccessDenied errors, refer to IAM Permissions for Installation.

  • The jumpbox can connect to the required registry. If you are not already authenticated, log in to it.

    • To authenticate to the Protegrity Container Registry (PCR), use the following command with credentials from the My.Protegrity portal during account creation:
    echo "<password>" | helm registry login registry.protegrity.com --username "<username>" --password-stdin
    
    • For a local registry, use your credentials and local registry endpoint as required.

EKS Cluster Requirements

Ensure that the EKS cluster meets the following requirements:

  • Kubernetes 1.27 or later is installed.
  • The default StorageClass is used unless you override it. Verify the StorageClass with kubectl get storageclass.
  • The EKS Pod Identity Agent add-on is enabled on the cluster.
  • Karpenter 1.0 or later is installed and configured on the cluster.

IAM Permissions for Installation

The AWS credentials used during installation require the following permissions. These permissions are required only during installation and can be reduced afterward.

ServicePermissionsPurpose
EKSeks:DescribeCluster, eks:DescribeAddon, eks:CreatePodIdentityAssociation, eks:DeletePodIdentityAssociation, eks:DescribePodIdentityAssociationVerify the cluster and its add-ons, and associate the IAM role with the anonymization pods through EKS Pod Identity.
IAMiam:CreateRole, iam:DeleteRole, iam:GetRole, iam:CreatePolicy, iam:DeletePolicy, iam:GetPolicy, iam:AttachRolePolicy, iam:DetachRolePolicyCreate and manage the IAM role and policy that grant the anonymization workers access to the S3 bucket.
S3s3:CreateBucket, s3:DeleteBucket, s3:PutBucketPolicy, s3:GetBucketPolicy, s3:PutObject, s3:GetObject, s3:DeleteObjectCreate and configure the S3 bucket that stores input and output datasets.

2 - Installing Protegrity Anonymization

Steps to install Protegrity Anonymization

Note: This guide uses version 2.0.1 throughout. Replace 2.0.1 with the version you are installing, including in the Python SDK Installation section.

This project deploys the Protegrity Anonymization service on Amazon EKS as part of the Protegrity AI Team Edition.

Deployment Steps

1. Provision Cloud Infrastructure with OpenTofu

The OpenTofu module creates the S3 bucket, IAM role, EKS Pod Identity association, and Karpenter NodePool. It also writes anonymization-values.yaml with cloud-specific Helm overrides.

Option A - You already have a root module

Add the following module block to your existing root module:

module "anonymization" {
  source = "oci://registry.protegrity.com/anonymization/anonymization-server/opentofu/anonymization?tag=aws-2.0.1"

  cluster_name  = "<CLUSTER_NAME>"
  bucket_name   = "<globally-unique-s3-bucket-name>"
  oci_host      = "registry.protegrity.com"
  environment   = "production"
  chart_version = "2.0.1"
}

Note: This module requires the hashicorp/aws (~> 5.0) and hashicorp/kubernetes (~> 2.35) providers.

If your root module does not already declare them, add the following to your terraform {} block:

required_providers {
  aws = {
    source  = "registry.opentofu.org/hashicorp/aws"
    version = "~> 5.0"
  }
  kubernetes = {
    source  = "registry.opentofu.org/hashicorp/kubernetes"
    version = "~> 2.35"
  }
}

Configure the kubernetes provider to connect to the target EKS cluster:

data "aws_eks_cluster" "this" {
  name = "<your-cluster-name>"
}

data "aws_eks_cluster_auth" "this" {
  name = "<your-cluster-name>"
}

provider "kubernetes" {
  host                   = data.aws_eks_cluster.this.endpoint
  cluster_ca_certificate = base64decode(data.aws_eks_cluster.this.certificate_authority[0].data)
  token                  = data.aws_eks_cluster_auth.this.token
}

Then run the following commands:

tofu init
tofu plan
tofu apply
Option B - You do not have a root module

Create a new directory with a single main.tf:

terraform {
  required_version = "~> 1.10"

  required_providers {
    aws = {
      source  = "registry.opentofu.org/hashicorp/aws"
      version = "~> 5.0"
    }
    kubernetes = {
      source  = "registry.opentofu.org/hashicorp/kubernetes"
      version = "~> 2.35"
    }
  }
}

variable "cluster_name" {
  type        = string
  description = "EKS cluster name."
}

variable "environment" {
  type    = string
  default = "production"
}

data "aws_eks_cluster" "this" {
  name = var.cluster_name
}

data "aws_eks_cluster_auth" "this" {
  name = var.cluster_name
}

provider "kubernetes" {
  host                   = data.aws_eks_cluster.this.endpoint
  cluster_ca_certificate = base64decode(data.aws_eks_cluster.this.certificate_authority[0].data)
  token                  = data.aws_eks_cluster_auth.this.token
}

module "anonymization" {
  source = "oci://registry.protegrity.com/anonymization/anonymization-server/opentofu/anonymization?tag=aws-2.0.1"

  cluster_name  = var.cluster_name
  bucket_name   = "<globally-unique-s3-bucket-name>"
  oci_host      = "registry.protegrity.com"
  environment   = var.environment
  chart_version = "2.0.1"
}

output "helm_install" {
  value = module.anonymization.helm_install
}

Then run the following commands:

tofu init
tofu plan -var="cluster_name=<cluster-name>" -var="bucket_name=<globally-unique-bucket>"
tofu apply -var="cluster_name=<cluster-name>" -var="bucket_name=<globally-unique-bucket>"

Note: tofu output -raw helm_install prints a ready-to-run Helm command with the correct version and namespace.

2. Deploy with Helm

The OpenTofu module writes an anonymization-values.yaml file in your working directory. Passwords are required only for the first installation. They are preserved automatically for later upgrades.

export MLOPS_PASSWORD=<MLOPS_PASSWORD>
export JOBSTATE_PASSWORD=<JOBSTATE_PASSWORD>

helm upgrade --install anonymization \
  oci://registry.protegrity.com/anonymization/anonymization-server/helm/anonymization \
  --version 2.0.1 \
  --namespace anon-ns --create-namespace \
  --values anonymization-values.yaml \
  --set database.auth.mlopsPassword=$MLOPS_PASSWORD \
  --set database.auth.jobstatePassword=$JOBSTATE_PASSWORD

Note: Ensure that “<JOBSTATE_PASSWORD>” contains only alphanumeric characters.

For upgrades, omit the --set flags because the passwords are already stored in the cluster secret:

helm upgrade anonymization \
  oci://registry.protegrity.com/anonymization/anonymization-server/helm/anonymization \
  --version 2.0.1 \
  --namespace anon-ns \
  --values anonymization-values.yaml

3. Monitor

  1. Monitor the deployment process using the following command.

    kubectl get pods -n anon-ns
    

    Verify that all pods are in the Running state. The following is a sample output.

    NAME                                  READY   STATUS    RESTARTS   AGE
    anon-app-depl-f5c4d4cd6-42wgn         1/1     Running   0          3m20s
    anon-db-depl-0                        1/1     Running   0          3m20s
    anon-scheduler-depl-7b87fcb74-l5q6v   1/1     Running   0          3m20s
    anon-worker-depl-7c4d95496f-djw7f     1/1     Running   0          3m20s
    anon-worker-depl-7c4d95496f-gnnvp     1/1     Running   0          3m20s
    
  2. Verify that the Anonymization service is deployed.

    kubectl get svc -n anon-ns
    

    The following is the sample output.

    NAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
    anon-app-svc    ClusterIP   172.20.151.139   <none>        8000/TCP   61s
    anon-dask-svc   ClusterIP   172.20.224.133   <none>        8786/TCP   61s
    

    For more information about the Python SDK, refer to Python SDK Reference.

3 - Configuring Protegrity Anonymization

Steps to configure Protegrity Anonymization.

Warning: Use the -k flag only in development or trusted internal environments. Do not use it in production, where valid certificates must be configured.

Role Update and User Creation

After deployment, update the default anonymization_administrator role to include the can_create_token permission and then create a user with this role.

1. Update anonymization_administrator role permission

export GATEWAY_URL="https://$(kubectl get configmap/nfa-config -n default -o jsonpath='{.data.FQDN}')"

# 1. Obtain an Authentication Token
TOKEN=$(curl -sk -X POST "${GATEWAY_URL}/api/v1/auth/login/token" \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'loginname=admin&password=Admin123!' \
  -D - -o /dev/null | grep -i 'pty_access_jwt_token' | awk '{print $2}' | tr -d '\r\n')

curl -sk -X PUT \
  "${GATEWAY_URL}/pty/v1/auth/roles" \
  -H 'accept: application/json' \
  -H "Authorization: Bearer ${TOKEN}" \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "anonymization_administrator",
    "description": "Administrator role",
    "permissions": [
      "can_create_token",
      "anonymization_operations_admin"
    ]
  }'

2. Create a user with the anonymization_administrator role

Use the following request payload when creating the user:

{
  "username": "anonymization_admin",
  "email": "anonadmin@example.com",
  "firstName": "Anon",
  "lastName": "<USERNAME>",
  "password": "<PASSWORD>",
  "roles": [
    "anonymization_administrator"
  ]
}

Create the user with the following API call:

curl -sk -X POST \
  "${GATEWAY_URL}/pty/v1/auth/users" \
  -H 'accept: application/json' \
  -H "Authorization: Bearer ${TOKEN}" \
  -H 'Content-Type: application/json' \
  -d '{
    "username": "anonymization_admin",
    "email": "anonadmin@example.com",
    "firstName": "Anon",
    "lastName": "<USERNAME>",
    "password": "<PASSWORD>",
    "roles": [
      "anonymization_administrator"
    ]
  }'

Note: In production environments, replace the default credentials and URLs. For example, for <USERNAME>, enter User and for <PASSWORD>, enter StrongPassword123!.

4 - Protegrity Anonymization Python SDK Installation

Steps to install Python SDK.

Use this section to install, configure, and validate the Python SDK for Protegrity Anonymization.

Prerequisites

Ensure one of the following is met:

  • pip is available in your active Python virtual environment.
  • A conda environment is activated.

Overview

Use the Python SDK to access the Anonymization service programmatically.

1. Install the Anonymization Python SDK

Install the SDK using pip or conda using the version that matches the deployed Anonymization service.

Using pip:

pip install protegrity-anonymization-sdk==2.0.1

Using conda:

conda install -c protegrity protegrity-anonymization-sdk==2.0.1

2. Configure the SDK

Create a ClientConfig pointing to your PPC gateway. Obtain the gateway URL using the following command:

export GATEWAY_URL="https://$(kubectl get configmap/nfa-config -n default -o jsonpath='{.data.FQDN}')"

Then configure the client in Python:

from anonymization_sdk.config import ClientConfig

config = ClientConfig(
    base_url="<GATEWAY_URL>/pty/anonymization/v3",
    username="<USERNAME>",
    password="<PASSWORD>",
    verify_ssl=True,
)

Alternatively, configure using environment variables:

export ANON_API_URL="<GATEWAY_URL>/pty/anonymization/v3"
export ANON_USERNAME="anonymization_admin"
export ANON_PASSWORD="StrongPassword123!"
from anonymization_sdk.config import ClientConfig

config = ClientConfig.from_env()

Note: Replace default credentials and URLs for production environments. For example, for <USERNAME>, enter anonymization_admin and for <PASSWORD>, enter StrongPassword123!.

3. Test the Anonymization Python SDK

from anonymization_sdk import AnonymizationClient
from anonymization_sdk.config import ClientConfig

config = ClientConfig(
    base_url="<GATEWAY_URL>/pty/anonymization/v3",
    username="anonymization_admin",
    password="StrongPassword123!",
    verify_ssl=True,
)

with AnonymizationClient(config=config) as client:
    print("Connection established successfully.")

The connection is established successfully. If the connection fails, an error message is displayed. For more information about SDK Reference, refer to Python SDK Reference.

5 - Uninstallation and Cleanup Protegrity Anonymization

Steps to uninstall and cleanup Protegrity Anonymization

To remove the Anonymization service and all associated Kubernetes and AWS resources:

1. Uninstall the Helm release

# Removes all chart resources. PVC data is RETAINED by default (retentionPolicy: whenDeleted=Retain).
helm uninstall anonymization -n anon-ns

# To also delete the database PVC:
kubectl delete pvc -n anon-ns -l app.kubernetes.io/name=anonymization

Optionally, delete the namespace after the Helm release and PVCs are removed using the following command:

kubectl delete namespace anon-ns

2. Destroy OpenTofu infrastructure

Warning: tofu destroy deletes the S3 bucket, including all stored input and output datasets. Back up any data you need before proceeding.

# Destroys the Karpenter nodes, IAM role, Pod Identity association, and S3 bucket.
tofu destroy -var="cluster_name=<CLUSTER_NAME>"

Note: By default, tofu destroy prompts for confirmation before deleting resources. Add -auto-approve to skip the prompt.