Telemetry Metrics on K8s

For K8s Environment

Akeyless Gateway Telemetry Metrics can be consumed by well-known monitoring and alerting solutions, such as Datadog or Prometheus. You can find a full list of supported endpoints on the official page of the Open Telemetry project.

The Telemetry Metrics are based on time series telemetry data metrics from the application and the runtime environment, storing them in a unique database or index, and analyzing data trends over time.

The metrics visualization uses a pre-made/custom dashboard (Grafana marketplace dashboard, Datadog integration dashboard, etc.).

The following Metrics are currently available:

MetricDescription
akeyless.gw.system.cpu.*CPU utilization metrics
akeyless.gw.system.disk.*Disk I/O metrics
akeyless.gw.system.load.*CPU load metrics
akeyless.gw.system.memory.*Memory utilization metrics
akeyless.gw.system.network.*Network interface I/O metrics & TCP connection metrics
akeyless.gw.quota.current_transactions_numberThe current total transaction count in the account
akeyless.gw.quota.gw_admin_client_transactionsTotal transactions made by the Gateway default identity (AdminAccessID)
akeyless.gw.quota.total_transactions_limitTotal transaction limit per hour in the account.
akeyless.gw.system.http_response_status_codeStatus of HTTP response for any request that originates from the Gateway API. (i.e. performed against the Gateway)
akeyless.gw.system.request_countTotal number of requests that were issued directly against the Gateway API (the count of total HTTP status)
akeyless.gw.system.healthcheck.statusMonitors container health check status

Datadog

To enable Telemetry Metrics on your Gateway for Datadog, edit the chart values.yaml file under the metrics section and set your metrics backend configuration:

metrics:
  enabled: true  
  config: |
    exporters:    
      datadog:
        api:
          key: "<Your Datadog API key>"
          site: <Your Datadog server site>
    service:
      pipelines:
        metrics:
          exporters: [datadog]

Set the relevant API Key of your Datadog server, and set the relevant site. If your Datadog server is running in the EU site, add site:datadoghq.eu by default it is set to the US site. If you did this before deploying your Gateway, go to Dashboard Setup. If you are adding this to a running Gateway, update your Gateway once done and continue to Dashboard Setup.

Dashboard Setup:

Akeyless is an official Datadog Partner and our dashboard can be found inside the Datadog app.

  • Go directly to your Datadog account and click on Integrations --> Integrations. Then choose Akeyless Gateway from the list of Integrations and click the Install Integration button.

  • Once installed, go to Dashboards --> Dashboard List and choose the Akeyless GW Dashboard that was installed.

  • If your Gateway metrics are up and running properly, you will see your Gateway metrics in the Akeyless GW dashboard. You can also go to the Metrics Explorer to see more metrics to add to the Dashboard by filtering for "akeyless.gw".

Prometheus

To enable Telemetry Metrics on your Gateway for Prometheus, edit the chart values.yaml file under the metrics section and set your metrics backend configuration:

expose the port 8889 (or any other port) for Prometheus Exporter and the Prometheus Scraping as described below:

service:
  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/port: "8889"


metrics:
  enabled: true  
  config: |
    exporters:
      prometheus:
        endpoint: "0.0.0.0:8889"
    service:
      pipelines:
        metrics:
          exporters: [prometheus]

Add a scraping target for the Akeyless Gateway in your Prometheus config file and restart your Prometheus server.

scrape_configs:
  - job_name: 'akeyless'
    scrape_interval: 10s
    static_configs:
    # docker for linux
      - targets: ['localhost:8889'] # for docker on macOS use['host.docker.internal:8889']

Once done, check your Prometheus server for the ingested metrics.

Grafana Dashboard

You can visualize Akeyless metrics in Grafana Dashboard when using Prometheus as a data source.

Import the Akeykess GW dashboard for your Grafana instance using this link.

Using K8s Secret

create a K8s secret configuration and save it to a file config-secret.yaml:

exporters:
  datadog:
    api:
      key: <api-key>      
service:
  pipelines:
    metrics:
      exporters: [datadog]
exporters:
  prometheus:
    endpoint: "0.0.0.0:8889"
service:
  pipelines:
    metrics:
      exporters: [prometheus]

Encode the file to base64:

base64 --input=config-secret.yaml

Create a k8s secret using the encoded value and place it in the data.otel-config.yaml field:

apiVersion: v1
kind: Secret
metadata:
  name: gw-metrics-secret
  namespace: <your-namespace>
type: Opaque
data:
  otel-config.yaml: <Base64 K8s Secret value>

Deploy the secret on your k8s cluster, and make sure to deploy the secret to the correct namespace:

kubectl apply -f secret.yaml -n <your-namespace>

Set your k8s secret name on the metrics.existingSecretName field in the Gateway chart values.yaml file:

metrics:
  enabled: true
  existingSecretName: "gw-metrics-secret"