Protegrity Synthetic Data generates privacy-safe synthetic datasets from real data using machine learning models such as VineCopula, TabDiff, TabularGAN, and SMOTE. The generated data is statistically representative and suitable for development, testing, and analytics workflows.
This is the multi-page printable view of this section. Click here to print.
Protegrity Synthetic Data
- 1: Introduction
- 2: Understanding the Architecture and Components
- 3: Prerequisites for Deployment
- 4: Server Installation
- 5: User Creation in PPC
- 6: Using Synthetic Data
- 6.1: Using REST
- 6.1.1: REST API Reference
- 6.2: Using Python SDK
- 6.2.1: Installation
- 6.2.2: Python SDK Reference
- 7: Uninstallation and Cleanup
- 8: Troubleshooting
1 - Introduction
This section explains why organizations use Synthetic Data and how Protegrity Synthetic Data preserves statistical utility while reducing privacy risk.
Organizations need realistic datasets for model training, testing, and cross-team collaboration without exposing sensitive information. Traditional methods such as masking and redaction can reduce data utility for analytics and machine learning.
Protegrity Synthetic Data learns patterns and distributions from source data. It then generates new records that reflect these patterns without reproducing the original personal records.
Synthetic data enables safer data sharing, faster development and validation, and compliance with regulations such as GDPR and HIPAA.
2 - Understanding the Architecture and Components
Protegrity Synthetic Data generates realistic, privacy-safe synthetic datasets using machine learning models. It learns patterns and distributions from real data, then produces new records with similar statistical properties.
Protegrity Synthetic Data uses Kubernetes for scalable Synthetic Data generation and supports deployment on AWS EKS.
Component Diagram
The following diagram shows the deployment architecture and communication between components.
graph TD
User([User / Client Application]) -->|HTTPS :443| GW[PPC Gateway]
User --> SDK[Synthetic Data Python SDK]
SDK -->|HTTPS :443| GW
GW -->|:8000 /pty/syntheticdata/v2| App[Synthetic Data
REST Server + ML Engine]
App -->|:5432 jobstatedb + mlopsdb| DB[(PostgreSQL)]
subgraph EKS [EKS Cluster · synthetic-data-ns]
GW
App
DB
end
subgraph AWS
S3[(S3 Bucket
Data Storage)]
IAM[IAM Role
Pod Identity]
end
App -.->|assumes| IAM
IAM -.->|grants access| S3
App <-->|read / write| S3Communication Ports
| Port | Direction | Description |
|---|---|---|
| 443 | User and SDK → PPC Gateway | HTTPS entry point for user and SDK traffic |
| 8000 | Gateway → Synthetic Data REST Server | Synthetic Data REST API (/pty/syntheticdata/v2) |
| 5432 | Synthetic Data REST Server → PostgreSQL | Internal connection for job state (jobstatedb) and MLOps model tracking (mlopsdb) |
Components
PPC Gateway: Exposes the external HTTPS endpoint and routes requests to the Synthetic Data REST API.
Synthetic Data REST Server: Exposes the
/pty/syntheticdata/v2API and runs ML models such as VineCopula, TabDiff, TabularGAN, and SMOTE in-process to produce synthetic datasets. It stores job metadata in PostgreSQL, reads input data from the S3 bucket and writes output data to it.PostgreSQL: Hosts two logical databases.
jobstatedbstores job metadata, including generation parameters, job status, and result references.mlopsdbstores MLOps model contracts, training runs, metrics, and artifact references managed by the MLOps library.S3 Storage: Stores input datasets and generated synthetic datasets.
IAM Role with EKS Pod Identity: Grants the Synthetic Data REST Server secure access to the S3 bucket without static AWS credentials.
Python SDK: Provides programmatic access to the REST API. For more information about the Python SDK, refer to the Python SDK Installation section.
3 - Prerequisites for Deployment
For more information about the tools and permissions required for deployment, refer to Prerequisites.
4 - Server Installation
For more information about installation steps, refer to Installation.
5 - User Creation in PPC
For more information about creating user in PPC, refer to Configuring Protegrity Synthetic Data.
6 - Using Synthetic Data
This section provides a consolidated view of Python SDK usage and API capabilities. It covers client configuration patterns and core synthetic data workflows such as health checks, quasi-identifier detection, and risk analysis.
6.1 - Using REST
6.1.1 - REST API Reference
Base URL (all schema-documented APIs): <GATEWAY_ENDPOINT>/pty/syntheticdata/v2
Authentication
- AUTH Key: Needed for client authentication
- Header:
Authorization: Bearer <AUTH_KEY>
- Header:
Common request models
DataInput
Provide exactly one of the following:
inline(base64 CSV string)uri(cloud URI such ass3://...,gs://...,azure://...,minio://...)inline_tables(multi-table map: table name -> base64 CSV)uri_tables(multi-table map: table name -> cloud URI)
Optional:
format:csvorparquet(used for inline payloads)
Error model (common behavior)
422validation error for schema/field violations403tier-gated feature not allowed on current tier429rate limit exceeded501async job tracking not enabled (when server runs without job store)
Async job submission response
Most write endpoints return 202 Accepted immediately:
{
"job_id": "1a2b3c4d-...",
"status": "queued",
"message": "Job submitted for background processing",
"created_at": "2026-08-04T06:00:00Z"
}
Use the Jobs endpoints to poll completion and fetch context.result.
1. Submit Synthesis Job
POST /synthesize
Submits synthesis operations such as fit, fit_transform, transform, summary, evaluate, validate_relationships, relational_score, get_table_order.
Request body
{
"model_name": "vine",
"action": "fit_transform",
"training_data": {
"inline": "<BASE64_CSV>"
},
"n_samples": 100,
"parameters": {
"categorical_cols": ["city"]
},
"output": {
"uri": "s3://my-bucket/synth/output.csv"
},
"post_filters": {
"business_rules": {
"intervals": { "age": [18, 65] },
"unique_combinations": [["country", "region"]]
}
},
"pre_filters": {
"outlier_detection": {
"contamination": 0.05
}
}
}
cURL
curl -X POST "<GATEWAY_ENDPOINT>/pty/syntheticdata/v2/synthesize" \
-H "Content-Type: application/json" \
-d '{
"model_name":"vine",
"action":"fit_transform",
"training_data":{"inline":"<BASE64_CSV>"},
"n_samples":100
}'
Response
202 Accepted->JobResponse
2. Submit Privacy Evaluation Job
POST /evaluate/privacy
Evaluates privacy risk (membership inference, sensitive attribute reconstruction, linkage-related checks).
Request body
{
"train_real_data": { "inline": "<BASE64_CSV>" },
"test_real_data": { "inline": "<BASE64_CSV>" },
"synthetic_data": { "inline": "<BASE64_CSV>" },
"sensitive_columns": ["diagnosis", "income"],
"k_values": [2, 5, 10],
"config": {
"shadow_models": 3,
"attack_model": "xgboost",
"random_state": 42
}
}
cURL
curl -X POST "<GATEWAY_ENDPOINT>/pty/syntheticdata/v2/evaluate/privacy" \
-H "Content-Type: application/json" \
-d '{
"train_real_data":{"inline":"<BASE64_CSV>"},
"test_real_data":{"inline":"<BASE64_CSV>"},
"synthetic_data":{"inline":"<BASE64_CSV>"}
}'
Response
202 Accepted->JobResponse- Final job result (
context.result) followsPrivacyEvaluationResponse
3. Submit Causal Fidelity Evaluation Job
POST /evaluate/causal
Runs one or more causal fidelity analyses: treatment effect, decision consistency, fairness shift.
Request body
{
"real_data": { "inline": "<BASE64_CSV>" },
"synthetic_data": { "inline": "<BASE64_CSV>" },
"treatment_col": "treatment",
"outcome_col": "outcome",
"covariates": ["age", "income"],
"target_col": "label",
"feature_cols": ["age", "income", "score"],
"task_type": "classification",
"sensitive_attr": "gender",
"config": {
"ate_threshold": 0.15,
"random_state": 123
}
}
cURL
curl -X POST "<GATEWAY_ENDPOINT>/pty/syntheticdata/v2/evaluate/causal" \
-H "Content-Type: application/json" \
-d '{
"real_data":{"inline":"<BASE64_CSV>"},
"synthetic_data":{"inline":"<BASE64_CSV>"},
"treatment_col":"treatment",
"outcome_col":"outcome"
}'
Response
202 Accepted->JobResponse- Final job result (
context.result) followsCausalEvaluationResponse
4. Submit Certification Job
POST /certify
Computes overall certification score and component breakdown (fidelity, privacy, utility, completeness).
Request body
{
"real_data": { "inline": "<BASE64_CSV>" },
"synthetic_data": { "inline": "<BASE64_CSV>" },
"categorical_cols": ["region", "product"],
"target_col": "purchased",
"task_type": "classification",
"include_privacy_attacks": true,
"train_real_data": { "inline": "<BASE64_CSV>" },
"test_real_data": { "inline": "<BASE64_CSV>" },
"feature_cols": ["age", "income"],
"sensitive_col": "diagnosis",
"quasi_identifiers": ["zipcode", "age", "gender"],
"fidelity_weight": 0.4,
"privacy_weight": 0.3,
"utility_weight": 0.2,
"completeness_weight": 0.1
}
cURL
curl -X POST "<GATEWAY_ENDPOINT>/pty/syntheticdata/v2/certify" \
-H "Content-Type: application/json" \
-d '{
"real_data":{"inline":"<BASE64_CSV>"},
"synthetic_data":{"inline":"<BASE64_CSV>"}
}'
Response
202 Accepted->JobResponse- Final job result (
context.result) followsCertificationResponse
5. Submit Conditional Generation Job
POST /generate/conditional
Generates synthetic data conditioned on filters and optional drift injection.
Request body
{
"real_data": { "inline": "<BASE64_CSV>" },
"model_name": "vine",
"categorical_cols": ["status", "fraud"],
"n_samples": 50,
"conditions": {
"fraud": 1,
"age": ">50",
"status": "active"
},
"amplify_patterns": {
"fraud": 2.0
},
"inject_drift": {
"income": -10000,
"age": -5
},
"random_state": 42
}
cURL
curl -X POST "<GATEWAY_ENDPOINT>/pty/syntheticdata/v2/generate/conditional" \
-H "Content-Type: application/json" \
-d '{
"real_data":{"inline":"<BASE64_CSV>"},
"model_name":"vine",
"n_samples":50,
"conditions":{"fraud":1}
}'
Response
202 Accepted->JobResponse- Final job result (
context.result) followsConditionalResult
6. List Production Models
GET /models
Returns model versions currently in production stage.
Query parameters
model_type(optional): filter by algorithm class (for examplevine)all_metrics(optional, defaultfalse): include all logged metrics
cURL
curl "<GATEWAY_ENDPOINT>/pty/syntheticdata/v2/models?model_type=vine&all_metrics=true"
Response
200 OK- Body:
ProductionModelInfo[]
Example:
[
{
"model_name": "vine_v1",
"model_type": "vine",
"model_version": "v1",
"semantic_version": "2.0",
"stage": "Production",
"input_schema": {"age": "float", "salary": "float", "region": "string"},
"metrics": {"tabsyndex_overall": 0.627},
"registered_at": "2026-03-18T11:18:03+00:00"
}
]
7. Submit Horizontal Benchmark Job
POST /benchmark/horizontal
Benchmarks multiple models on one dataset.
Request body
Provide either dataset_name or custom data (+ categorical_columns, target_variable).
{
"data": { "inline": "<BASE64_CSV>" },
"categorical_columns": ["region"],
"target_variable": "purchased",
"models": ["smote", "tabdiff"],
"n_rows_override": 1000
}
cURL
curl -X POST "<GATEWAY_ENDPOINT>/pty/syntheticdata/v2/benchmark/horizontal" \
-H "Content-Type: application/json" \
-d '{
"dataset_name":"heart",
"models":["smote","tabdiff"]
}'
Response
202 Accepted->JobResponse- Final job result (
context.result) followsHorizontalBenchmarkResponse
8. Submit Vertical Benchmark Job
POST /benchmark/vertical
Benchmarks one model across all predefined hyperparameter presets.
Request body
Provide either dataset_name or custom data (+ categorical_columns, target_variable).
{
"data": { "inline": "<BASE64_CSV>" },
"categorical_columns": ["region"],
"target_variable": "purchased",
"model": "smote",
"n_rows_override": 1000
}
cURL
curl -X POST "<GATEWAY_ENDPOINT>/pty/syntheticdata/v2/benchmark/vertical" \
-H "Content-Type: application/json" \
-d '{
"dataset_name":"heart",
"model":"smote"
}'
Response
202 Accepted->JobResponse- Final job result (
context.result) followsVerticalBenchmarkResponse
9. Job APIs (available when job store is enabled)
These routes are mounted at /jobs via pty_ai_job_state_lib and are part of the public API surface when async job tracking is configured.
9.1 List jobs
GET /jobs
Optional query parameters observed in tests:
status(for examplecompleted)
curl "<GATEWAY_ENDPOINT>/pty/syntheticdata/v2/jobs?status=completed"
9.2 Get job details
GET /jobs/{job_id}
curl "<GATEWAY_ENDPOINT>/pty/syntheticdata/v2/jobs/<JOB_ID>"
Typical fields include job_id, status, message, progress, and context.
9.3 Get job history
GET /jobs/{job_id}/history
curl "<GATEWAY_ENDPOINT>/pty/syntheticdata/v2/jobs/<JOB_ID>/history"
9.4 Delete job
DELETE /jobs/{job_id}
curl -X DELETE "<GATEWAY_ENDPOINT>/pty/syntheticdata/v2/jobs/<JOB_ID>"
Expected: 204 No Content when deletion succeeds.
6.2 - Using Python SDK
6.2.1 - Installation
Use this section to install, configure, and validate the Python SDK for Protegrity Synthetic Data.
Prerequisites
Ensure one of the following is met:
pipis available in your active Python virtual environment.- A conda environment is activated.
Overview
The Synthetic Data service can be accessed programmatically using the Python SDK.
1. Install the Synthetic Data Python SDK
Install the SDK from PyPI or conda using the version that matches the deployed Synthetic Data service.
Using pip:
pip install protegrity-synthetic-data-sdk>=2.1.0
Using conda:
conda install -c protegrity protegrity-synthetic-data-sdk>=2.1.0
2. Configure the SDK
Create a client configuration pointing to your PPC gateway. The gateway URL can be obtained using:
export GATEWAY_URL="https://$(kubectl get configmap/nfa-config -n default -o jsonpath='{.data.FQDN}')"
Then configure the client in Python:
from synthetic_data_sdk.config import ClientConfig
config = ClientConfig(
base_url="<GATEWAY_URL>/pty/syntheticdata/v2",
username="<USERNAME>",
password="<PASSWORD>",
verify_ssl=True,
)
Note: Replace default credentials and URLs for production environments. For example, for
<USERNAME>, entersyntheticdata_adminand for<PASSWORD>, enterStrongPassword123!.
3. Test the Synthetic Data Python SDK
from synthetic_data_sdk import SyntheticDataClient
from synthetic_data_sdk.config import ClientConfig
config = ClientConfig(
base_url="<GATEWAY_URL>/pty/syntheticdata/v2",
username="<USERNAME>",
password="<PASSWORD>",
verify_ssl=True,
)
with SyntheticDataClient(config=config) as client:
print("Connection established successfully.")
The connection is established successfully. If the connection fails, an error message is displayed.
6.2.2 - Python SDK Reference
Protegrity AI (Developer or TEAM) Editions - This section documents all classes and methods available in Protegrity Synthetic Data TEAMS and later. Methods marked as (TEAMS) are available only in the AI TEAM edition and above.
ClientConfig
Configuration dataclass for all Synthetic Data SDK clients.
from synthetic_data_sdk import ClientConfig
Attributes:
- endpoint (
str) - Base URL of the Synthetic Data API (e.g."<GATEWAY_URL>/pty/syntheticdata/v2"). - timeout (
int, default300) - Request timeout in seconds. - max_retries (
int, default3) - Retry attempts on connection failure. - verify_ssl (
bool, defaultTrue) - Set toFalsefor self-signed certificates. - api_key (
str | None, defaultNone) - API key for authentication if required. - headers (
dict, default{}) - Additional HTTP headers for every request.
Single-Table Synthesizers
All single-table synthesizers share the same fit / transform / fit_transform / evaluate / summary interface, inherited from a shared base class.
RemoteVineCopula
Single-table vine copula synthesizer. Available in all tiers.
from synthetic_data_sdk import RemoteVineCopula
synth = RemoteVineCopula(
endpoint="<GATEWAY_URL>/pty/syntheticdata/v2",
categorical_cols=["city", "product"],
)
Constructor parameters:
- endpoint (
str) - API endpoint URL. Not required whenconfigis provided. - model_version (
str, optional) - Version identifier for model persistence. Reuse to avoid refitting. - config (
ClientConfig, optional) - Advanced client configuration. - mlops_config (
dict, optional) - Per-request MLOps tracking override. - **parameters - Model-specific hyper-parameters (e.g.
categorical_cols,vine_type).
RemoteSMOTE
SMOTE-based oversampling synthesizer. Useful for augmenting minority classes in imbalanced datasets. Available in all tiers.
from synthetic_data_sdk import RemoteSMOTE
synth = RemoteSMOTE(
endpoint="<GATEWAY_URL>/pty/syntheticdata/v2",
categorical_cols=["class"],
k=5,
)
Constructor parameters: Same as RemoteVineCopula. Model-specific parameters: categorical_cols, k, noise_scale.
RemoteTabDiff (TEAMS)
Diffusion-based single-table synthesizer. Ideal for GPU-intensive synthesis without local GPU resources. Requires TEAMS.
from synthetic_data_sdk import RemoteTabDiff
synth = RemoteTabDiff(
endpoint="<GATEWAY_URL>/pty/syntheticdata/v2",
categorical_cols=["city", "product"],
epochs=1000,
)
Constructor parameters: Same as RemoteVineCopula. Model-specific parameters: categorical_cols, epochs.
RemoteTabularGAN (TEAMS)
TabularGAN (CTABGAN architecture) synthesizer with mode-specific normalization for mixed continuous/categorical columns. Requires TEAMS.
from synthetic_data_sdk import RemoteTabularGAN
synth = RemoteTabularGAN(
endpoint="<GATEWAY_URL>/pty/syntheticdata/v2",
categorical_cols=["city", "product"],
epochs=300,
)
Constructor parameters: Same as RemoteVineCopula. Model-specific parameters: categorical_cols, epochs.
Shared Methods (all single-table synthesizers)
fit
def fit(df: pd.DataFrame | str | Path) -> Self
Fit the model on training data. The fitted model is stored on the server using the configured model_version.
Parameters:
- df (
DataFrame | str | Path) - Training data as a DataFrame, local file path, or cloud URI (s3://,gs://,azure://,minio://). Cloud URIs (s3://,gs://,azure://) require TEAMS.
Returns: Self (for method chaining).
Raises:
SynthesisAPIError- If fitting fails.
transform
def transform(n: int) -> pd.DataFrame
Generate synthetic data using a fitted model.
Parameters:
- n (
int) - Number of synthetic samples to generate.
Returns: DataFrame - Synthetic data with the same schema as the training data.
Raises:
RuntimeError- If the model has not been fitted and nomodel_versionexists on the server.SynthesisAPIError- If generation fails.
fit_transform
def fit_transform(df: pd.DataFrame, n: int) -> pd.DataFrame
Fit the model and generate synthetic data in a single call.
Parameters:
- df (
DataFrame) - Training data. - n (
int) - Number of synthetic samples to generate.
Returns: DataFrame - Synthetic data.
evaluate
def evaluate(
real_data: pd.DataFrame | str,
synthetic_data: pd.DataFrame | str,
categorical_cols: list[str] | None = None,
target_col: str | None = None,
task_type: str | None = None,
eval_params: dict[str, Any] | None = None,
) -> dict[str, Any]
Evaluate synthetic data quality against the real data.
Parameters:
- real_data (
DataFrame | str) - Real training data. - synthetic_data (
DataFrame | str) - Synthetic data to evaluate. - categorical_cols (
list[str], optional) - Categorical column names. - target_col (
str, optional) - Target column for TSTR/TRTR evaluation. - task_type (
str, optional) -"classification"or"regression"for predictive evaluation. - eval_params (
dict, optional) - AdditionalFidelityEvaluatorconfiguration.
Returns: dict - Evaluation metrics. Basic metrics (distributional, column-level utility, memorization) are available in all tiers. Advanced metrics (privacy attacks, causal fidelity, certification, rare-population) require TEAMS.
summary
def summary() -> dict[str, Any]
Get summary statistics from a fitted model.
Returns: dict - Model summary with statistics and metadata.
Raises:
RuntimeError- If the model has not been fitted.
transform_conditional (TEAMS - RemoteVineCopula only)
def transform_conditional(
df: pd.DataFrame,
n: int,
conditions: dict[str, Any] | None = None,
amplify_patterns: float | None = None,
inject_drift: dict[str, float] | None = None,
random_state: int | None = None,
) -> pd.DataFrame
Generate synthetic data matching specific conditional scenarios. Requires TEAMS (generation:conditional).
Parameters:
- df (
DataFrame) - Training data to fit on. - n (
int) - Number of synthetic samples to generate. - conditions (
dict, optional) - Column filter conditions:- Exact match:
{"fraud": 1, "status": "active"} - Comparison:
{"age": ">65", "income": "<=50000"} - Range:
{"age": "between(30,50)"} - Membership:
{"city": "in(NYC,LA,Chicago)"}
- Exact match:
- amplify_patterns (
float, optional) - Multiplier for conditional pattern amplification (e.g.1.5for 50% increase). - inject_drift (
dict, optional) - Column drift shifts (e.g.{"income": -20000}for recession scenario). - random_state (
int, optional) - Random seed for reproducibility.
Returns: DataFrame - Synthetic data matching the specified conditions.
Multi-Table Synthesizer
RemoteMultiTableVineCopula
Multi-table vine copula synthesizer. Preserves foreign-key relationships across tables. Available in all tiers.
from synthetic_data_sdk import RemoteMultiTableVineCopula
synth = RemoteMultiTableVineCopula(
endpoint="<GATEWAY_URL>/pty/syntheticdata/v2",
relationships=[
("customers", "customer_id", "orders", "customer_id"),
("orders", "order_id", "items", "order_id"),
],
)
tables = {"customers": customers_df, "orders": orders_df, "items": items_df}
synth.fit(tables)
synthetic = synth.transform(n=500)
Exposes the same fit, transform, fit_transform, evaluate, summary, and validate_relationships workflow as RemoteVineCopula, but accepts and returns dict[str, DataFrame] instead of DataFrame.
Evaluation Clients (TEAMS)
The following evaluation clients require TEAMS and provide advanced quality and privacy assessments beyond the metrics returned by evaluate().
PrivacyEvaluator (TEAMS)
Evaluates privacy risks in synthetic data using membership inference attacks, sensitive attribute reconstruction, and linkage attack risk analysis.
from synthetic_data_sdk import PrivacyEvaluator
evaluator = PrivacyEvaluator(endpoint="<GATEWAY_URL>/pty/syntheticdata/v2")
results = evaluator.evaluate(
train_real_data=train_df,
test_real_data=test_df,
synthetic_data=synthetic_df,
sensitive_columns=["ssn", "salary", "diagnosis"],
)
print(f"Overall Risk: {results['overall_risk']}")
evaluate(train_real_data, test_real_data, synthetic_data, sensitive_columns, k_values, config) → dict
- train_real_data - Real training data used to generate synthetic data.
- test_real_data - Held-out real data (not seen during training).
- synthetic_data - Synthetic data to evaluate.
- sensitive_columns (
list[str], optional) - Columns to test for attribute inference. - k_values (
list[int], optional) - K values for linkage attack evaluation. - config (
dict, optional) - Attack configuration (e.g.{"shadow_models": 10, "attack_model": "xgboost"}).
Returns: dict - Contains overall_risk, attacks list, and summary with successful_attacks.
CertificationClient (TEAMS)
Produces a comprehensive certification score (0–100) with letter grade (A+ to F) by aggregating fidelity, privacy, utility, and completeness metrics.
Score components: Fidelity 40% · Privacy 30% · Utility 20% · Completeness 10%.
from synthetic_data_sdk import CertificationClient
cert = CertificationClient(endpoint="<GATEWAY_URL>/pty/syntheticdata/v2")
result = cert.certify(
real_data=real_df,
synthetic_data=synthetic_df,
categorical_cols=["city", "gender"],
target_col="income",
task_type="regression",
)
print(f"Grade: {result['grade']} Score: {result['overall_score']:.1f}/100")
certify(real_data, synthetic_data, categorical_cols, target_col, task_type, include_privacy_attacks, ...) → dict
Returns: dict - Contains grade (A+ to F), overall_score (0–100), risk_level, summary, and recommendations.
CausalEvaluator (TEAMS)
Evaluates whether synthetic data preserves causal relationships, decision boundaries, and fairness properties from the original real data.
from synthetic_data_sdk import CausalEvaluator
evaluator = CausalEvaluator(endpoint="<GATEWAY_URL>/pty/syntheticdata/v2")
results = evaluator.evaluate(
real_data=real_df,
synthetic_data=synthetic_df,
treatment_col="received_treatment",
outcome_col="recovery_time",
covariates=["age", "severity"],
)
print(f"Preservation Rate: {results['summary']['preservation_rate']:.1%}")
evaluate(real_data, synthetic_data, treatment_col, outcome_col, target_col, feature_cols, task_type, sensitive_attr, covariates) → dict
Returns: dict - Contains overall_preserved, evaluations list, and summary.
Low-Level Client
SynthesisClient
Low-level client for direct API interaction. Most users should prefer the high-level synthesizer classes above.
synthesize
def synthesize(
model_name: str,
action: str,
training_data: str | None = None,
training_data_path: str | None = None,
training_data_tables: dict[str, str] | None = None,
n_samples: int | None = None,
model_version: str | None = None,
parameters: dict[str, Any] | None = None,
output_uri: str | None = None,
mlops_config: dict[str, Any] | None = None,
) -> dict[str, Any]
Send a synthesis request (synchronous - waits for completion).
Parameters:
- model_name (
str) - Model type."vine","vine_multitable","smote"are available in all tiers."tabdiff"and"tabulargan"require TEAMS. - action (
str) - Action:"fit","transform", or"fit_transform". - training_data (
str, optional) - Base64-encoded CSV for single-table inline data. - training_data_path (
str, optional) - Cloud URI or local path.s3://,azure://,gcs://require TEAMS;minio://is available in all tiers. - training_data_tables (
dict[str, str], optional) - Table name → path/URI mapping for multi-table synthesis. - n_samples (
int, optional) - Number of synthetic samples to generate. - model_version (
str, optional) - Version identifier for model persistence. - parameters (
dict, optional) - Model-specific hyper-parameters. - output_uri (
str, optional) - Cloud URI for output. Same tier rules astraining_data_path. - mlops_config (
dict, optional) - Per-request MLOps tracking configuration.
Returns: dict - API response with status, data, and metadata.
Raises:
SynthesisAPIError- If the request fails.
synthesize_async
Same parameters as synthesize(). Returns immediately with {"job_id": "...", "status": "queued"}.
generate_conditional (TEAMS)
def generate_conditional(
real_data: str | pd.DataFrame,
model_name: str,
n_samples: int,
conditions: dict[str, Any] | None = None,
amplify_patterns: float | None = None,
inject_drift: dict[str, float] | None = None,
categorical_cols: list[str] | None = None,
random_state: int | None = None,
) -> dict[str, Any]
Generate conditional synthetic data via the low-level client. Requires TEAMS (generation:conditional).
Parameters:
- real_data (
str | DataFrame) - Training data. - model_name (
str) - One of"vine","smote","tabdiff"(TEAMS),"tabulargan"(TEAMS). - n_samples (
int) - Samples to generate. - conditions (
dict, optional) - Column filter conditions (seetransform_conditional). - amplify_patterns (
float, optional) - Pattern amplification multiplier. - inject_drift (
dict, optional) - Column drift shifts. - categorical_cols (
list[str], optional) - Categorical columns. - random_state (
int, optional) - Random seed.
Returns: dict - Contains success, n_samples, synthetic_data (base64 CSV), conditions_applied, drift_applied, warnings, and metadata.
Job Management
get_job_status
def get_job_status(job_id: str) -> dict[str, Any]
Get the current status of an asynchronous job.
Parameters:
- job_id (
str) - Job identifier returned bysynthesize_async().
Returns: dict - Contains job_id, status (pending, running, completed, failed, cancelled), progress, step, message, error, synth_data_uri, and timestamps.
wait_for_job
def wait_for_job(
job_id: str,
poll_interval: float = 2.0,
timeout: float = 600.0,
callback: Any | None = None,
) -> dict[str, Any]
Block until a job reaches a terminal state.
Parameters:
- job_id (
str) - Job identifier. - poll_interval (
float, default2.0) - Seconds between status polls. - timeout (
float, default600.0) - Maximum seconds to wait. - callback (
callable, optional) - Called with the current status dict after each poll.
Returns: dict - Final job status.
Raises:
TimeoutError- If the job does not complete withintimeout.
list_jobs
def list_jobs(
status: str | None = None,
limit: int = 100,
offset: int = 0,
) -> dict[str, Any]
List jobs with optional status filter and pagination.
Parameters:
- status (
str, optional) - Filter by status:"pending","running","completed","failed","cancelled". - limit (
int, default100) - Page size (1–1000). - offset (
int, default0) - Page offset.
Returns: dict - Contains jobs list, total, limit, and offset.
get_job_history
def get_job_history(job_id: str) -> list[dict[str, Any]]
Get the full state-transition audit trail for a job.
Parameters:
- job_id (
str) - Job identifier.
Returns: list[dict] - Each entry contains sequence, status, progress, step, and changed_at.
delete_job
def delete_job(job_id: str) -> None
Delete a job record, or cancel it if still running.
Parameters:
- job_id (
str) - Job identifier.
Exceptions (SynthesisAPIError) are importable from synthetic_data_sdk.exceptions.
7 - Uninstallation and Cleanup
For more information about removing the Synthetic Data service and all associated Kubernetes and AWS resources, refer to Uninstallation and Cleanup.
8 - Troubleshooting
Diagnostic Flowchart
Use the following flowchart to identify the cause of common issues.
flowchart TD
Start[Problem?] --> ServerRunning{Server pods Running?}
ServerRunning -->|No| PodIssue[Check pod logs<br/>kubectl logs -n synthetic-data-ns]
ServerRunning -->|Yes| APIWorks{Does API respond?}
APIWorks -->|No| NetworkIssue[Check Service and Gateway<br/>kubectl get svc -n synthetic-data-ns]
APIWorks -->|Yes| ValidationFails{Request fails?}
ValidationFails -->|Yes| ConfigIssue[Review request config<br/>Check model and data parameters]
ValidationFails -->|No| OtherIssue[Check logs or open a support ticket]
style Start fill:#e1f5ffCommon Issues
Pods not reach the Running state
kubectl describe pod -n synthetic-data-ns <POD_NAME>
kubectl logs -n synthetic-data-ns <POD_NAME>
Check that the Karpenter NodePool provisioned the required nodes. Ensure that OpenTofu created the S3 bucket and IAM role.
API returns 401 Unauthorized
Ensure that the login token is valid and has not expired. Re-authenticate using the login endpoint and update the Authorization header.
API returns 403 Forbidden
Verify that the user’s role includes the can_create_token permission. Follow the User Creation in PPC steps to confirm role configuration.
OpenTofu errors on tofu apply
AccessDenied: The AWS credentials used during installation do not have the required IAM permissions. For more information about permissions, refer to the IAM Permissions for Installation section.BucketAlreadyExists: An S3 bucket with the same name already exists. Choose a unique bucket name or import the existing bucket into the OpenTofu state.
Helm upgrade fails with immutable field error
Delete the existing release, then re-run the install command from Server Installation:
helm uninstall synthetic-data -n synthetic-data-ns
# Re-run the install command
For more troubleshooting guidance, refer to Protegrity Support.