Usage Metrics Logging
Usage metrics provide information about the usage of tunnels, services, profiles, and rules. By default, the usage metrics feature is enabled in the gateway.json file.
For more information about the gateway.json file, refer to the section Gateway.json file.
The following snippet shows how the usage metrics feature is enabled in the gateway.json file.
"stats": {
"enabled": true
}
The metrics are recorded in CSV format in the gateway.log file, and then parsed to JSON and sent to Insight.
For more information about viewing the usage metrics on Insight, refer to the section Forwarding Usage Metrics to the Audit Store.
The logs are emitted at a default interval of 120 seconds. During this 120-second window, all requests processed by the gateway process will be recorded. To modify the time window, configure the usageLogInterval parameter in the stats setting in gateway.json file.
Note:
The time interval is calculated once the gateway restarts.
The usage metrics will only be logged when there is a transaction.
The following table describes the usage metrics for Tunnels.
Parameter | Data Type in DSG | Data Type in Insight | Description | Example |
---|---|---|---|---|
metrics_type | string | string | The metric type is displayed as tunnel. | Tunnel |
version | integer | integer | A version for the tunnel metric type. | 0 |
tunnel_type | string | string | The type of tunnel used to process a request is displayed. | HTTP |
logtype | string | string | The value to identify type of metric such as dsg_metrics_usage_tunnel. | dsg_metrics_usage_tunnel |
log_time | epoch_millis | date | The time when the usage is reported. | 1707242725766 |
log_interval | integer | long | The time difference between the current and previous logs. | 30003 |
tunnel_id | string | string | The unique ID of the tunnel. | t-808715e0-b725-4781-8bf3-429220dd46d5 |
uptime | float | double | The time in seconds since the tunnel loaded. | 22670.46733236313 |
bytes_processed | integer | long | The frontend and backend bytes the tunnel processed since the last time usage was reported. | 38 |
frontend_bytes_processed | integer | long | The frontend bytes the tunnel has processed since the last time usage was reported. | 38 |
backend_bytes_processed | integer | long | The backend bytes the tunnel has processed since the last time usage was reported. | 0 |
total_bytes_processed | integer | long | The total number of frontend and backend bytes the tunnel has processed during the time the tunnel has been loaded. | 38 |
total_frontend_bytes_processed | integer | long | The total number of frontend bytes the tunnel has processed during the time the tunnel has been loaded. | 38 |
total_backend_bytes-_processed | integer | long | The total number of backend bytes the tunnel has processed during the time the tunnel has been loaded. | 0 |
message_count | integer | long | The number of requests the tunnel received since the last time usage was reported. | 1 |
total_message_count | integer | long | The total number of requests the tunnel received during the time the tunnel has been loaded. | 1 |
origin_time_utc* | NA | date | The time in UTC at which this log is ingested. | Feb 26, 2024 @ 03:51:54.416 |
* -The origin_time_utc parameter will only be displayed on the Audit Store Dashboards.
The following table describes the usage metrics for Services.
Parameter | Data Type in DSG | Data Type in Insight | Description | Example |
---|---|---|---|---|
metrics_type | string | string | The metric type is displayed as Service. | Service |
version | integer | integer | A version for the service metric type. | 0 |
service_type | string | string | The type of service used to process a request is displayed. | REST-API |
logtype | string | string | The value to identify type of metric such as dsg_metrics_usage_service. | dsg_metrics_usage_service |
log_time | epoch_millis | date | The time when the usage is reported. | 1707242 725766 |
log_interval | integer | long | The time difference between the current and previous logs. | 30003 |
service_id | string | string | The unique ID of the service. | s-62a3a161-6bd7-42fa-a9c6-6357d77824ca |
parent_id | string | string | The unique ID of the tunnel rule. | t-808715e0-b725-4781-8bf3-429220dd46d5 |
calls | integer | long | The number of times the service processed frontend and backend requests since the time the usage was last reported. | 38 |
frontend_calls | integer | long | The number of times the service processed frontend requests since the time the usage was last reported. | 38 |
backend_calls | integer | long | The number of times the service processed backend requests since the time the usage was last reported. | 0 |
total_calls | integer | long | The total number of times the service processed frontend and backend requests since the service has been loaded. | 38 |
total_frontend_calls | integer | long | The total number of times the service processed frontend and backend requests since the service has been loaded. | 38 |
total_backend_calls | integer | long | The total number of times the service processed frontend and backend requests since the service has been loaded. | 0 |
bytes_processed | integer | long | The frontend and backend bytes the service processed since the last time usage was reported. | 2 |
frontend_bytes_processed | integer | long | The frontend bytes the tunnel processed since the last time usage was reported. | 1 |
backend_bytes_processed | integer | long | The backend bytes the tunnel processed since the last time usage was reported. | 1 |
total_bytes_processed | integer | long | The total number of frontend and backend bytes the service has processed during the time the service has been loaded. | 2 |
total_frontend_bytes_processed | integer | long | The total number of frontend bytes the tunnel has processed during the time the tunnel has been loaded. | 1 |
total_backend_bytes_processed | integer | long | The total number of backend bytes the tunnel has processed during the time the tunnel has been loaded. | 1 |
origin_time_utc* | NA | date | The time in UTC at which this log is ingested. | Feb 26, 2024 @ 03:51:54.416 |
* -The origin_time_utc parameter will only be displayed on the Audit Store Dashboards.
The following table describes the usage metrics for Profile.
Parameter | Data Type in DSG | Data Type in Insight | Description | Example |
---|---|---|---|---|
metrics_type | string | string | The metric type is displayed as Profile. | Profile |
version | integer | integer | A version for the profile metric type. | 0 |
log_time | epoch_millis | date | The time when the usage is reported. | 1707242725766 |
log_interval | integer | long | The time difference between the current and previous logs. | 339439 |
logtype | string | string | The value to identify type of metric such as dsg_metrics_usage_profile. | dsg_metrics_usage_profile |
parent_id | string | string | The unique ID of the service rule. | s-62a3a161-6bd7-42fa-a9c6-6357d77824ca |
profile_id | string | string | The unique ID of the profile. | p-b335795f-8e77-4b15-9ba0-06002cc29bb9 |
calls | integer | long | The number of times the profile processed a request since the time usage was last reported. | 1 |
total_calls | integer | long | The total number of times the profile processed a request since profile has been loaded. | 1 |
profile_reference_count | integer | long | The number of times this profile has been called through a profile reference since the time the usage was last reported. | 0 |
total_profile_reference_count | integer | long | The total number of times this profile has been called through a profile reference since the profile has been loaded. | 0 |
bytes_processed | integer | long | The bytes the profile processed since the last time the usage was reported. | 38 |
total_bytes_processed | integer | long | The total bytes the profile processed since the profile has been loaded. | 38 |
elapsed_time_sample_count | integer | long | The number of times the profile was sampled since the last time the usage was reported. | 1 |
elapsed_time_average | integer | long | The average amount of time in nano-seconds it took to process a request based on elapsed-time-sample-count. | 13172454 |
total_elapsed_time_sample_count | integer | long | The number of times the profile was sampled since the profile has been loaded. | 1 |
total_elapsed_time_sample_average | integer | long | The average amount of time in nano-seconds it took to process a request based on total-elapsed-time-sample-count. | 13172454 |
origin_time_utc* | NA | date | The time in UTC at which this log is ingested. | Feb 26, 2024 @ 03:51:54.420 |
* -The origin_time_utc parameter will only be displayed on the Audit Store Dashboards.
The following table describes the usage metrics for Rules.
Name | Data Type in DSG | Data Type in Insight | Description | Example |
---|---|---|---|---|
metrics version | integer | integer | A version for the rule metric type. | 0 |
rule-type | string | string | The type of rule used to process a request is displayed. | Extract |
codec | string | string | It will display the type of payload extracted or the method used for data transformation. | Text |
logtype | string | string | The value to identify type of metric such as dsg_metrics_usage_rule. | dsg_metrics_usage_rule |
log_time | epoch_millis | date | The time when the usage is reported. | 1707242725766 |
log_interval | date | date | The time difference between the current and previous logs. | 22670424 |
broken | boolean | boolean | It indicates whether the rule is broken or not. | false |
domain_name_rewrite | boolean | boolean | It indicates whether the rule is domain name rewrite or not. | false |
rule_id | string | string | The unique ID of the rule. | r-38b72f16-f838-4602-81aa-cd881a76e418 |
parent_id | string | string | The unique ID of the profile rule. | p-b335795f-8e77-4b15-9ba0-06002cc29bb9 |
calls | integer | long | The number of times the rule processed a request since the time the usage was last reported. | 1 |
total_calls | integer | long | The total number of times the rule processed a request since rule has been loaded. | 1 |
profile_reference_count | integer | long | The number of times this rule has been called via a profile reference since the time the usage was last reported. | 0 |
total_profile_reference_count | integer | long | The total number of times this rule has been called via a profile reference since the rule has been loaded. | 0 |
bytes_processed | integer | long | The bytes the rule processed since the last time the usage was reported. | 1 |
total_bytes_processed | integer | long | The total bytes the rule processed since the rule has been loaded. | 1 |
elapsed_time_sample_count | integer | long | The number of times the rule was sampled since the last time usage was reported. | 1 |
elapsed_time_sample_average | integer | long | The average amount of time in nano-seconds it took to process a data based on elapsed-time-sample-count. | 13137378 |
total_elapsed_time_sample_count | integer | long | The number of times the rule was sampled since the rule has been loaded. | 1 |
total_elapsed_time_sample_average | integer | long | The average amount of time in nano-seconds it took to process a data based on total-elapsed-time-sample-count. | 13137378 |
origin_time_utc* | NA | date | The time in UTC at which this log is ingested. | Feb 26, 2024 @ 03:51:54.420 |
* -The origin_time_utc parameter will only be displayed on the Audit Store Dashboards.
Forwarding Usage Metrics to the Audit Store
The usage metrics is also forwarded to the Audit Store and can be viewed on the Audit Store Dashboards.
Ensure that the following prerequisites are met before you view the logs on the Audit Store Dashboards:
The Analytics component is initialized on the ESA. The initialization of Analytics is required for displaying the Audit Store information in the Audit Store Dashboards.
For more information about initializing the Analytics, refer to the section Initializing analytics on the ESA in the Protegrity Installation Guide.
For more information about the audit indexes, refer to the section Understanding the audit index fields in the Protegrity Insight Guide.
The logs are forwarded to the Audit Store.
For more information about forwarding the logs, refer to the section Forwarding Audit Logs to the Audit Store.
The following figure shows the sample usage metrics on the Discover screen of the Audit Store Dashboards.
Note: The index_node, tiebreaker, and index_time_utc parameters are only logged on the Audit Store Dashboards.
For more information about these parameters, refer to the section Understanding the audit index fields in the Protegrity Insight Guide.
The DSG usage metrics logs are stored in the pty_insight_analytics_dsg_usage_metrics_9.2 index file. You can configure and enable the scheduled task to free up the space used by old index files that you do not require. For usage metrics, edit the Delete DSG Usage Indices task and enable the task. The scheduled task can be set to n days based on your preference.
For more information about scheduled tasks, refer to the section Using the scheduler in the Protegrity Insight Guide 9.2.0.0.