Telemetry Metrics on K8s

For K8s Environment

Akeyless Gateway Telemetry Metrics can be consumed by well-known monitoring and alerting solutions, such as Datadog or Prometheus.

The Telemetry Metrics are based on time series telemetry data metrics from the application and the runtime environment, storing them in a unique database or index, and analyzing data trends over time.

The metrics visualization uses a pre-made/custom dashboard (Grafana marketplace dashboard, Datadog integration dashboard, etc.).

The following Metrics are currently available:

MetricDescription
akeyless.gw.system.cpu.*CPU utilization metrics
akeyless.gw.system.disk.*Disk I/O metrics
akeyless.gw.system.load.*CPU load metrics
akeyless.gw.system.memory.*Memory utilization metrics
akeyless.gw.system.network.*Network interface I/O metrics & TCP connection metrics
akeyless.gw.quota.current_transactions_numberCurrent transaction number
akeyless.gw.quota.gw_admin_client_transactionsTotal transaction by an admin client
akeyless.gw.quota.total_transactions_limitTotal transaction limit per hour
akeyless.gw.system.http_response_status_codeStatus of HTTP response
akeyless.gw.system.request_countTotal number of requests

Datadog

To enable Telemetry Metrics on your Gateway for Datadog, edit the chart values.yaml file under the metrics section and set your metrics backend configuration:

metrics:
  enabled: true  
  config: |
    exporters:    
      datadog:
        api:
          key: "<Your Datadog API key>"
          site: <Your Datadog server site>
    service:
      pipelines:
        metrics:
          exporters: [datadog]

Set the relevant API Key of your Datadog server, and set the relevant site. If your Datadog server is running in the EU site, add site:datadoghq.eu by default it is set to the US site.

Dashboard Setup:

  • Download the dashboard JSON file from here.

  • In the Datadog navigation pane, create a new dashboard, Dashboards > New Dashboards.

  • Go to settings (on the top right) and choose Import dashboard JSON and select the downloaded JSON file.

Upon successful setup of the Gateway, navigate to the Datadog platform, and filter the Akeyless metrics on the summary page: https://app.datadoghq.com/metric/summary?filter=akeyless.gw

Prometheus

To enable Telemetry Metrics on your Gateway for Prometheus, edit the chart values.yaml file under the metrics section and set your metrics backend configuration:

expose the port 8889 (or any other port) for Prometheus Exporter and the Prometheus Scraping as described below:

metrics:
  enabled: true  
  config: |
    exporters:
      prometheus:
        endpoint: "0.0.0.0:8889"
    service:
      pipelines:
        metrics:
          exporters: [prometheus]

Add a scraping target for the Akeyless Gateway in your Prometheus config file and restart your Prometheus server.

scrape_configs:
  - job_name: 'akeyless'
    scrape_interval: 10s
    static_configs:
    # docker for linux
      - targets: ['localhost:8889'] # for docker on macOS use['host.docker.internal:8889']

Once done, check your Prometheus server for the ingested metrics.

Grafana Dashboard

You can visualize Akeyless metrics in Grafana Dashboard when using Prometheus as a data source.

Import the Akeykess GW dashboard for your Grafana instance using this link.

Using K8s Secret

create a K8s secret configuration and save it to a file config-secret.yaml:

exporters:
  datadog:
    api:
      key: <api-key>      
service:
  pipelines:
    metrics:
      exporters: [datadog]
exporters:
  prometheus:
    endpoint: "0.0.0.0:8889"
service:
  pipelines:
    metrics:
      exporters: [prometheus]

Encode the file to base64:

base64 --input=config-secret.yaml

Create a k8s secret using the encoded value and place it in the data.otel-config.yaml field:

apiVersion: v1
kind: Secret
metadata:
  name: gw-metrics-secret
  namespace: <your-namespace>
type: Opaque
data:
  otel-config.yaml: <Base64 K8s Secret value>

Deploy the secret on your k8s cluster, and make sure to deploy the secret to the correct namespace:

kubectl apply -f secret.yaml -n <your-namespace>

Set your k8s secret name on the metrics.existingSecretName field in the Gateway chart values.yaml file:

metrics:
  enabled: true
  existingSecretName: "gw-metrics-secret"