Concurrency
Guidance on concurrency.
Concurrency
Snowflake provides guidance on the maximum concurrent requests to the Protegrity API. However, reaching this maximum request depends on additional factors, such as, cluster use and available resources. In addition, depending on the query plan, individual batches may be processed serially across different UDFs.
The formula for theoretical maximum Snowflake concurrency is N * C * M * E * P:
- N - # of servers in the cluster (e.g. 2xl = 32, xl = 16)
- C - # of CPUs. This is typically 8, but depends on the hardware.
- M – parallelism multiplier (fixed to 8)
- E - # of external functions invoked
- P - # of queries in running in parallel
The following table shows this calculation for a single query.
| Cluster size | Predicted concurrent per query * | 1 UDF | 2 UDF | 5 UDF | 10 UDF |
|---|---|---|---|---|---|
| Medium | 4 servers x 8 CPU x 8 = 256 | 256 | 512 | 1,280 | 2,560 |
| X-Large | 16 servers x 8 CPU x 8 = 1,024 | 1,024 | 2,048 | 5,120 | 10,240 |
| 2X-Large | 32 servers x 8 CPU x 8 = 2,048 | 2,048 | 4,096 | 10,240 | 20,480 |
Note
* theoretical maximum concurrent requests based on engineering guidance from Snowflake.Feedback
Was this page helpful?