⌨️CLI

The agent offers a CLI for conveniently modifying attributes and connections.

The agent stores its configuration in the hidden .datasentinel directory within the user's home directory, using two files: agent.yml and connections.yml. These files can be manually managed if necessary.

Prerequisites

Enable the execution of the CLI

Using an alias

alias datasentinel='DATASENTINEL_PATH=<<INSTALL_DIRECTORY>>/datasentinel;LD_LIBRARY_PATH=$DATASENTINEL_PATH/lib/ $DATASENTINEL_PATH/datasentinel'

Using environment variables

export DATASENTINEL_PATH="<<INSTALL_DIRECTORY>>/datasentinel"
export LD_LIBRARY_PATH=$DATASENTINEL_PATH/lib:$LD_LIBRARY_PATH
export PATH=$DATASENTINEL_PATH:$PATH

Agent Management

Show Commands

Simply enter the datasentinel command.

datasentinel

Response

Copyright 2023 (c) Datasentinel- All rights reserved        www.datasentinel.io
================================================================================

  Agent:
        - start agent
        - stop agent
        - status agent
        - show metrics
                Show internal agent metrics and cache activity
        - set port <port number>
        - set collection-rate <low|high>
                low: The sessions collection is done every 10 seconds
                high: The sessions collection is done every second  (Default value)
                If the datasentinel extension is not installed, the collection-rate is automatically adjusted to low value
        - set tables-monitoring-limit (default 1000)
                The agent monitors the activity of tables and indexes if the number of tables in the connection is less than the defined limit
        - set query-monitoring-limit (default 10000)
                The agent tracks query activity, storing unique IDs in a daily cache. Exceeding limit triggers automatic deactivation of query monitoring.

  Connections: when the connections are disabled, the agent is disconnected.
        - enable all
        - disable all

  Connection:
        - add connection <name> -f <json file>
        - update connection <name> -f <json file>
                json example: {
                                "host": "hostname",
                                "port": 4587,
                                "user": "username",
                                "password": "value",
                                "tags": "key=value,key=value,..."
                              }

        - update connection <name> samples <on|off> (default off)
                Collect and send sample queries, with literal values if present

        - update connection <name> lock-monitoring <on|off> (default on)
                Scan blocking sessions (longer than lock-monitoring-delay) and send blocking reports

        - update connection <name> lock-monitoring-delay <seconds> (default 30)
                Minimum number of seconds before triggering a blocking scenario

        - update connection <name> table-monitoring <on|off> (default on)
                Monitor the activity of tables and indexes (The number of tables must be lower than the maximum limit set at the agent level (default 1000)

        - update connection <name> query-monitoring <on|off> (default on)
                Monitor pg_stat_statements query activity

        - update connection <name> query-monitoring-optimize <on|off> (default on)
                Group similar pg_stat_statements queries

        - update connection <name> query-monitoring-min-calls <calls> (default 2)
        - update connection <name> query-monitoring-min-time <seconds> (default 1)
                Retrieve pg_stat_statements queries executed n times (calls) or with total execution time exceeding specified seconds

        - update connection <name> test-query-monitoring-optimize
                Test and optimize pg_stat_statements query contents with previous parameters, display results

        - delete connection <name>
        - enable connection <name>
        - disable connection <name>

        - show connections
        - show connection <name>

    Dump connection in JSON format:
        - dump connection <name>

    Upload server:
        - set server <host> <port>
        - show server
        - test server

    Token:
        - set token <value>
        - show token

    Proxy:
        - set proxy -f <json file>
                json example: {
                                "host": "hostname",
                                "port": 4587,
                                "user": "username (optional)",
                                "password": "value (optional)"
                              }
        - delete proxy
        - show proxy

Status

datasentinel status agent

Response

Copyright 2023 (c) Datasentinel- All rights reserved        www.datasentinel.io
================================================================================

              Agent
                  Version : 3.2.0
                  Server : pgsentinel.localdomain
                    Port : 8282
              Start time : 2023-01-31 13:08:27
          Collection rate : high
  Table monitoring limit : 2500
  Query monitoring limit : 35000
            Sql max size : 7512453

              Proxy
                    host :
                    port : 0
                    user :
                password :

              Upload
                    host : 163.172.135.154
                    port : 443

        Connections
                declared : 2
                  running : 1
              not running : 1

Start

datasentinel start agent

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

Starting the agent...


To show the status : datasentinel status agent

Stop

datasentinel stop agent

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

The agent is stopped!

Change Port Number

By default, the agent is configured to listen on port 8282, enabling API or CLI calls to be made.

datasentinel set port 9121

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

Agent port successfully set! The agent has been stopped, you need to restart it

Change Table Monitoring Limit

datasentinel set tables-monitoring-limit <limit>

Parameter:
The limit number. The agent tracks the activity of tables and indexes if the connection’s table count is below the set limit (default 1000).

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

Tables monitoring limit successfully set to 3000!


                Agent
                    Version : 3.2.0
                    Server : pgsentinel.localdomain
                      Port : 8282
                Start time : 2023-01-31 13:08:27
            Collection rate : low
    Table monitoring limit : 2500
    Query monitoring limit : 35000
              Sql max size : 7512453

                Proxy
                      host :
                      port : 0
                      user :
                  password :

                Upload
                      host : 163.172.135.154
                      port : 443

          Connections
                  declared : 2
                    running : 1
                not running : 1

Change Query monitoring Limit

Since agent version 3.2

datasentinel set query-monitoring-limit <limit>

Parameter:
The limit number. The agent tracks query activity, storing unique IDs in a daily cache. Exceeding limit triggers automatic deactivation of query monitoring. (default 30000)

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

Tables monitoring limit successfully set to 3000!


                Agent
                    Version : 3.2.0
                    Server : pgsentinel.localdomain
                      Port : 8282
                Start time : 2023-01-31 13:08:27
            Collection rate : low
    Table monitoring limit : 2500
    Query monitoring limit : 35000
              Sql max size : 7512453

                Proxy
                      host :
                      port : 0
                      user :
                  password :

                Upload
                      host : 163.172.135.154
                      port : 443

          Connections
                  declared : 2
                    running : 1
                not running : 1

Upload Server

The Upload Server refers to the server name of the platform where metrics are transmitted.

Set

Change the server where metrics are sent

datasentinel set server app.datasentinel.io 443

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

Server successfully set!


        Server
                host : app.datasentinel.io
                port : 443

Show

Show the platform server where metrics are sent

datasentinel show server

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

        Server
                host : app.datasentinel.io
                port : 443

Test

datasentinel test server

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

TEST: The upload server is reachable and up!


        Server
                host : app.datasentinel.io
                port : 443

Token

Set

Configure the authentication token used to access the platform. This should only be done during initialization. The agent subsequently retrieves updated tokens from the platform.

datasentinel set token <token value>

Response

Copyright 2023 (c) Datasentinel- All rights reserved        www.datasentinel.io
================================================================================

Token successfully set!


          Server
                  host : app.datasentinel.io
                  port : 443

Show

After testing the token on the platform's upload server, the validity date of the token is provided.

datasentinel show token

Response

Copyright 2023 (c) Datasentinel- All rights reserved        www.datasentinel.io
================================================================================

                    Token : <token value>

            Organization : ds-data
          Expiration Date : 2023-07-22 18:09:29

Proxy

Under certain circumstances, an intermediary proxy server is required to establish a connection between the agent and the platform server.

Set

create a json file with proxy settings and update the agent

JSON description

cat > proxy.json << EOF
{
    "host": "myProxyHostName",
    "port": 4587,
    "user": "username (optional)",
    "password": "value (optional)"
}
EOF

datasentinel set proxy -f proxy.json

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

Proxy successfully set!


          Proxy
                host : myProxyHostName
                port : 4587
                user :
            password :

Show

datasentinel show proxy

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

          Proxy
                host : hostname
                port : 4587
                user :
            password :

Delete

datasentinel delete proxy

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

Proxy successfully deleted!


          Proxy
                host :
                port : 0
                user :
            password :

Connections

Show

datasentinel show connections

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

name                 status     state      host                 port   user
--------------------------------------------------------------------------------
:9342                enabled    running    pg-sales-1734          9342 datasentinel
  tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Enable All

datasentinel enable all

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

Connections enabled!


name                 status     state      host                 port   user
--------------------------------------------------------------------------------
:9342                enabled    running    pg-sales-1734          9342 datasentinel
  tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Disable All

datasentinel disable all

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

Connections disabled!


name                 status     state      host                 port   user
--------------------------------------------------------------------------------
:9342                disabled   not running pg-sales-1734          9342 datasentinel
  tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Connection

Add

Create a JSON file with connection settings

JSON

cat > newConnection.json << EOF
{
    "host": "pg-sales-1734",
    "port": 9342,
    "user": "datasentinel",
    "password": "sentinel",
    "tags": "application=sales,environment=production,provider=amazon,datacenter=lyon",
}
EOF

datasentinel add connection myNewConnection -f newConnection.json

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

Connection added!


                   Name : myNewConnection
                 Status : enabled
                  State : connected

                   Host : pg-sales-1734
                   Port : 9342

                   user : datasentinel
               password : sentinel

        Lock monitoring : on
  Lock monitoring delay : 30s
       Table monitoring : on

          Query samples : off

       Query monitoring : on
               Optimize : on
              Min calls : 2
               Min time : 1s

                   tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Show

datasentinel show connection myNewConnection

Parameter
The connection name

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

                   Name : myNewConnection
                 Status : enabled
                  State : connected

                   Host : pg-sales-1734
                   Port : 9342

                   user : datasentinel
               password : sentinel

        Lock monitoring : on
  Lock monitoring delay : 30s
       Table monitoring : on

          Query samples : off

       Query monitoring : on
               Optimize : on
              Min calls : 2
               Min time : 1s

                   tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Dump

Dump connection properties in JSON format

datasentinel dump connection myNewConnection

Parameter
The connection name

Response

{
    "name": "myNewConnection",
    "host": "pg-sales-1734",
    "port": 9342,
    "user": "datasentinel",
    "password": "sentinel",
    "tags": "application=sales,environment=production,provider=amazon,datacenter=lyon ",
    "enabled": true,
    "connected": true,
    "collection-rate": "low",
    "samples": false,
    "lock-monitoring": true,
    "lock-monitoring-delay": 30,
    "table-monitoring": true,
    "query-monitoring": true,
    "query-monitoring-optimize": true,
    "query-monitoring-min-calls": 2,
    "query-monitoring-min-time": 1
}

Delete

datasentinel delete connection myNewConnection

Parameter
The connection name

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

Connection successfully deleted!

Enable

Activate the connection and initiate connection attempts in order to start monitoring

datasentinel enable connection myNewConnection

Parameter
The connection name

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

Connection enabled!


                  Name : myNewConnection
                Status : enabled
                State : connected

                  Host : pg-sales-1734
                  Port : 9342

                  user : datasentinel
              password : sentinel

      Lock monitoring : on
Lock monitoring delay : 30s
      Table monitoring : on

        Query samples : off

      Query monitoring : on
              Optimize : on
            Min calls : 2
              Min time : 1s

                  tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Disable

datasentinel disable connection myNewConnection

Parameter
The connection name

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

Connection disabled!


                  Name : myNewConnection
                Status : enabled
                State : connected

                  Host : pg-sales-1734
                  Port : 9342

                  user : datasentinel
              password : sentinel

      Lock monitoring : on
Lock monitoring delay : 30s
      Table monitoring : on

        Query samples : off

      Query monitoring : on
              Optimize : on
            Min calls : 2
              Min time : 1s

                  tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Update

cat > updateConnection.json << EOF
{
    "host": "pg-sales-1734",
    "port": 9342,
    "user": "datasentinel",
    "password": "sentinel",
    "tags": "application=sales,environment=production,provider=microsoft,datacenter=paris",
}
EOF

datasentinel update connection myNewConnection -f updateConnection.json

Parameter
The connection name

Response

Copyright 2023 (c) datasentinel- All rights reserved        www.datasentinel.io
================================================================================

Connection updated!


                    Name : myNewConnection
                  Status : enabled
                  State : connected

                    Host : pg-sales-1734
                    Port : 9342

                    user : datasentinel
                password : sentinel

        Lock monitoring : on
  Lock monitoring delay : 30s
        Table monitoring : on

          Query samples : off

        Query monitoring : on
                Optimize : on
              Min calls : 2
                Min time : 1s

                    tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Collection Level

The Agent Version 3.2 introduces the Collection Level feature, which was released in February 2023 and allows you to choose the metrics to monitor.

Query Samples

datasentinel update connection ":9342" samples on|off

Parameter
The connection name

Lock Monitoring

datasentinel update connection ":9342" lock-monitoring on|off

Parameter
The connection name

Lock Monitoring Delay

datasentinel update connection ":9342" lock-monitoring-delay <seconds>

Parameters
The connection name Seconds

Table Monitoring

datasentinel update connection ":9342" table-monitoring on|off

Parameter
The connection name

Query Monitoring

datasentinel update connection ":9342" query-monitoring on|off

Parameter
The connection name

Query Monitoring Optimize

datasentinel update connection ":9342" query-monitoring-optimize on|off

Parameter
The connection name

Query Monitoring Min Calls

datasentinel update connection ":9342" query-monitoring-min-calls <value>

Parameters
The connection name Minimum number of calls registered in pg_stat_statements

Query Monitoring Min Time

datasentinel update connection ":9342" query-monitoring-min-time <seconds>

Parameter
The connection name Minimum total execution time (seconds)

Internals

Since Agent Version 3.2

Show Metrics

Displays internal metrics and caches for debugging purposes

datasentinel show metrics

JSON output

{
  "version": "3.2.0",
  "port": 8282,
  "start_time": "2023-01-31 13:08:27",
  "memory": "59MB",
  "threads": 18,
  "pg_connections": 2,
  "running": 1,
  "not_running": 1,
  "connections": [
    {
      "name": "test",
      "stat_statements": 0,
      "stat_statements_cache": 0,
      "stat_plans_cache": 0,
      "query_ids_cache": 0,
      "missing_query_ids": 0,
      "plan_ids_cache": 0,
      "cache_tables": 0,
      "table_stats_cache": 0,
      "index_stats_cache": 0,
      "normal_task_runtime": 0,
      "low_task_runtime": 0,
      "tables_monitoring_limit": 2500,
      "query_monitoring": true,
      "query_monitoring_limit": 35000,
      "query_monitoring_min_calls": 2,
      "query_monitoring_min_time": 1,
      "query_monitoring_optimize": true,
      "xa_trans_optimized": 0,
      "insert_values_optimized": 0,
      "in_list_query_optimized": 0,
      "execution_plan_optimized": 0,
      "samples": false,
      "lock_monitoring": true,
      "lock_monitoring_delay": 10,
      "pool_used": 0
    },
    {
      "name": "test_connection",
      "stat_statements": 109,
      "stat_statements_cache": 109,
      "stat_plans_cache": 52,
      "query_ids_cache": 51,
      "missing_query_ids": 0,
      "plan_ids_cache": 48,
      "cache_tables": 1,
      "table_stats_cache": 148,
      "index_stats_cache": 352,
      "normal_task_runtime": 1663,
      "low_task_runtime": 12069,
      "tables_monitoring_limit": 2500,
      "query_monitoring": true,
      "query_monitoring_limit": 35000,
      "query_monitoring_min_calls": 21,
      "query_monitoring_min_time": 33,
      "query_monitoring_optimize": true,
      "xa_trans_optimized": 0,
      "insert_values_optimized": 23,
      "in_list_query_optimized": 1,
      "execution_plan_optimized": 0,
      "samples": true,
      "lock_monitoring": true,
      "lock_monitoring_delay": 10,
      "pool_used": 3
    }
  ]
}

Test Query Monitoring Optimization

The Query Optimize feature fetches queries from pg_stat_statements, filtered by minimum calls or minimum total execution time, groups similar queries, and shows the optimization gain at each level.

datasentinel update connection ":9342" test-query-monitoring-optimize

Parameter
The connection name

JSON output

{
  "pg_stat_statements": 964,
  "min_calls_filter": 21,
  "min_time_filter": "33s",
  "filtered_statements_get_time": "153ms",
  "filtered_statements_with_min_calls_or_min_time": 109,
  "filtered_ratio": "89%",
  "optimized_statements": 109,
  "optimized_ratio": "0%",
  "optimized_statements_run_time": "8ms"
}

PreviousRelease Notes NextAPI

Last updated 1 year ago