Performance on

Performance Considerations

Mon, 01 Jan 0001 00:00:00 +0000

Performance Considerations

The following factors may cause variation in real performance versus benchmarks:

Cold startup: The Lambda spends additional time on the initial invocation to decrypt and load the policy into memory. This time can vary between 400 ms and 1200 ms depending on the policy size. Once the Lambda is initialized, subsequent “warm executions” should process quickly.
Size of policy: The size of the policy impacts cold start performance. Larger policies take more time to initialize.
Noisy neighbors: There are many multi-tenant components in the Cloud. The same request may differ by 50% between runs regardless of Protegrity. A single execution may not be the best predictor of average performance.
Lambda memory: AWS provides more virtual cores based on the memory configuration. The initial configuration of 1728 MB provides a good tradeoff between performance and cost with the benchmarked policy. Memory can be increased to optimize for your individual cases.
Cluster size: Cluster size may make a significant difference depending on the workload.
Number of operations Number of protect, unprotect and reprotect security operations.
Lambda concurrency and burst quotas: AWS limits the number of concurrent executions and how quickly lambda can scale to meet demand. This is discussed in an upcoming section of the document.
Size of data element: Operations on larger text consume time.

Sample Benchmarks

Mon, 01 Jan 0001 00:00:00 +0000

Sample Benchmarks

The following benchmarks were performed against different cluster sizes. These are median times of approximately five runs each. The query unprotected six columns per row (first_name, last_name, email, postal_code, ssn, iban):

Rows x Cols	# Ops	Small	Medium	Large	XL	2XL
100K x 6 cols	600K	5.1	4	4.2	4.3	4.3
1M x 6 cols	6M	11	10	11	10	11
10M x 6 cols	60M	21.5	21.5	20.5	24.5	29
100M x 6 cols	600M	-	-	49.5	56.5	87

Concurrency

Mon, 01 Jan 0001 00:00:00 +0000

Concurrency

Snowflake provides guidance on the maximum concurrent requests to the Protegrity API. However, reaching this maximum request depends on additional factors, such as, cluster use and available resources. In addition, depending on the query plan, individual batches may be processed serially across different UDFs.

The formula for theoretical maximum Snowflake concurrency is N * C * M * E * P:

N - # of servers in the cluster (e.g. 2xl = 32, xl = 16)
C - # of CPUs. This is typically 8, but depends on the hardware.
M – parallelism multiplier (fixed to 8)
E - # of external functions invoked
P - # of queries in running in parallel

The following table shows this calculation for a single query.

Concurrency Tuning

Mon, 01 Jan 0001 00:00:00 +0000

Lambda Tuning

AWS maintains quotas for Lambda concurrent execution. Two of these quotas impact concurrency and compete with other Lambdas in the same account and region:

Log Forwarder Performance Tuning

Mon, 01 Jan 0001 00:00:00 +0000

Log Forwarder Performance

Log forwarder architecture is optimized to minimize the amount of connections and reduce the overall network bandwidth required to send audit logs to ESA. This is achieved with batching and aggregation taking place on two levels. The first level is in protect function instances, where audit logs from consecutive requests to an instance are batched and aggregated. The second level of batching takes place in Amazon Kinesis Stream where log records from different protect function instances are additionally batched and sent to log forwarder function where they are aggregated. This section shows how to configure the deployment to accommodate different patterns of anticipated audit log stream. It also shows how to monitor deployment resources to detect problems before audit records are lost.

Mon, 01 Jan 0001 00:00:00 +0000

API Gateway Tuning

AWS maintains a Throttle quota for the API Gateway. By default, API Gateway limits concurrent requests to 10,000 requests per second and throttles subsequent traffic with a 429 Too Many Requests error response. This quota applies across all APIs in an account and region.

The API Gateway default quota may need to be increased based on the Concurrency table in Lambda Tuning. Keep in mind that hitting quota limits effectively throttles any other API services in the region.

Mon, 01 Jan 0001 00:00:00 +0000

Cold-Start Performance

Cold-start vs warm execution refers to the state of the Lambda when a request is received. A cold-start undergoes additional initialization, such as, loading the security policy. Warm execution applies to all subsequent requests served by the Lambda. The following table shows an example how these states impact latency and performance.

Execution state	Avg. Execution Duration	Avg. Total (w/ network latency)
Cold execution	438 ms	522 ms
Warm execution	< 2ms	84 ms

Note

Cold execution time will vary based on the physical size of the security policy. A large security policy will result in longer cold startup times.

Mon, 01 Jan 0001 00:00:00 +0000

Concurrency Troubleshooting

Hitting up against quota limits may indicate that quota adjustments are required. Exceeding quota limits may cause a Snowflake query to fail or reduce performance. In the worst case, significant throttling can impact the performance of all your API Gateway or Lambda services in the region.

Snowflake is tolerant of a certain ratio of failed requests and automatically retries. If a high percentage of requests fail, then the query may abort and the last error code will display in the console. For example, 429 Too Many Requests.

Mon, 01 Jan 0001 00:00:00 +0000

Lambda Tuning

AWS maintains quotas for Lambda concurrent execution. Two of these quotas impact concurrency and compete with other Lambdas in the same account and region:

The concurrent executions quota cap is the maximum number of Lambda instances that can serve requests for an account and region. The default AWS quota may be inadequate based on peak concurrency based on the table in the previous section. This quota can be increased with an AWS support ticket.