Links
Comment on page

CLI

The agent offers a CLI for conveniently modifying attributes and connections.
The agent stores its configuration in the hidden .datasentinel directory within the user's home directory, using two files: agent.yml and connections.yml. These files can be manually managed if necessary.

Prerequisites

Enable the execution of the CLI
  • Using an alias
alias datasentinel='DATASENTINEL_PATH=<<INSTALL_DIRECTORY>>/datasentinel;LD_LIBRARY_PATH=$DATASENTINEL_PATH/lib/ $DATASENTINEL_PATH/datasentinel'
  • Using environment variables
export DATASENTINEL_PATH="<<INSTALL_DIRECTORY>>/datasentinel"
export LD_LIBRARY_PATH=$DATASENTINEL_PATH/lib:$LD_LIBRARY_PATH
export PATH=$DATASENTINEL_PATH:$PATH

Agent Management

Show Commands

  • Simply enter the datasentinel command.
datasentinel
Response
Copyright 2023 (c) Datasentinel- All rights reserved www.datasentinel.io
================================================================================
Agent:
- start agent
- stop agent
- status agent
- show metrics
Show internal agent metrics and cache activity
- set port <port number>
- set collection-rate <low|high>
low: The sessions collection is done every 10 seconds
high: The sessions collection is done every second (Default value)
If the datasentinel extension is not installed, the collection-rate is automatically adjusted to low value
- set tables-monitoring-limit (default 1000)
The agent monitors the activity of tables and indexes if the number of tables in the connection is less than the defined limit
- set query-monitoring-limit (default 10000)
The agent tracks query activity, storing unique IDs in a daily cache. Exceeding limit triggers automatic deactivation of query monitoring.
Connections: when the connections are disabled, the agent is disconnected.
- enable all
- disable all
Connection:
- add connection <name> -f <json file>
- update connection <name> -f <json file>
json example: {
"host": "hostname",
"port": 4587,
"user": "username",
"password": "value",
"tags": "key=value,key=value,..."
}
- update connection <name> samples <on|off> (default off)
Collect and send sample queries, with literal values if present
- update connection <name> lock-monitoring <on|off> (default on)
Scan blocking sessions (longer than lock-monitoring-delay) and send blocking reports
- update connection <name> lock-monitoring-delay <seconds> (default 30)
Minimum number of seconds before triggering a blocking scenario
- update connection <name> table-monitoring <on|off> (default on)
Monitor the activity of tables and indexes (The number of tables must be lower than the maximum limit set at the agent level (default 1000)
- update connection <name> query-monitoring <on|off> (default on)
Monitor pg_stat_statements query activity
- update connection <name> query-monitoring-optimize <on|off> (default on)
Group similar pg_stat_statements queries
- update connection <name> query-monitoring-min-calls <calls> (default 2)
- update connection <name> query-monitoring-min-time <seconds> (default 1)
Retrieve pg_stat_statements queries executed n times (calls) or with total execution time exceeding specified seconds
- update connection <name> test-query-monitoring-optimize
Test and optimize pg_stat_statements query contents with previous parameters, display results
- delete connection <name>
- enable connection <name>
- disable connection <name>
- show connections
- show connection <name>
Dump connection in JSON format:
- dump connection <name>
Upload server:
- set server <host> <port>
- show server
- test server
Token:
- set token <value>
- show token
Proxy:
- set proxy -f <json file>
json example: {
"host": "hostname",
"port": 4587,
"user": "username (optional)",
"password": "value (optional)"
}
- delete proxy
- show proxy

Status

datasentinel status agent
Response
Copyright 2023 (c) Datasentinel- All rights reserved www.datasentinel.io
================================================================================
Agent
Version : 3.2.0
Server : pgsentinel.localdomain
Port : 8282
Start time : 2023-01-31 13:08:27
Collection rate : high
Table monitoring limit : 2500
Query monitoring limit : 35000
Sql max size : 7512453
Proxy
host :
port : 0
user :
password :
Upload
host : 163.172.135.154
port : 443
Connections
declared : 2
running : 1
not running : 1

Start

datasentinel start agent
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Starting the agent...
To show the status : datasentinel status agent

Stop

datasentinel stop agent
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
The agent is stopped!

Change Port Number

By default, the agent is configured to listen on port 8282, enabling API or CLI calls to be made.
datasentinel set port 9121
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Agent port successfully set! The agent has been stopped, you need to restart it

Change Table Monitoring Limit

datasentinel set tables-monitoring-limit <limit>
  • Parameter:
    The limit number. The agent tracks the activity of tables and indexes if the connection’s table count is below the set limit (default 1000).
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Tables monitoring limit successfully set to 3000!
Agent
Version : 3.2.0
Server : pgsentinel.localdomain
Port : 8282
Start time : 2023-01-31 13:08:27
Collection rate : low
Table monitoring limit : 2500
Query monitoring limit : 35000
Sql max size : 7512453
Proxy
host :
port : 0
user :
password :
Upload
host : 163.172.135.154
port : 443
Connections
declared : 2
running : 1
not running : 1

Change Query monitoring Limit

Since agent version 3.2
datasentinel set query-monitoring-limit <limit>
  • Parameter:
    The limit number. The agent tracks query activity, storing unique IDs in a daily cache. Exceeding limit triggers automatic deactivation of query monitoring. (default 30000)
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Tables monitoring limit successfully set to 3000!
Agent
Version : 3.2.0
Server : pgsentinel.localdomain
Port : 8282
Start time : 2023-01-31 13:08:27
Collection rate : low
Table monitoring limit : 2500
Query monitoring limit : 35000
Sql max size : 7512453
Proxy
host :
port : 0
user :
password :
Upload
host : 163.172.135.154
port : 443
Connections
declared : 2
running : 1
not running : 1

Upload Server

The Upload Server refers to the server name of the platform where metrics are transmitted.

Set

  • Change the server where metrics are sent
datasentinel set server app.datasentinel.io 443
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Server successfully set!
Server
host : app.datasentinel.io
port : 443

Show

Show the platform server where metrics are sent
datasentinel show server
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Server
host : app.datasentinel.io
port : 443

Test

datasentinel test server
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
TEST: The upload server is reachable and up!
Server
host : app.datasentinel.io
port : 443

Token

Set

Configure the authentication token used to access the platform. This should only be done during initialization. The agent subsequently retrieves updated tokens from the platform.
datasentinel set token <token value>
Response
Copyright 2023 (c) Datasentinel- All rights reserved www.datasentinel.io
================================================================================
Token successfully set!
Server
host : app.datasentinel.io
port : 443

Show

After testing the token on the platform's upload server, the validity date of the token is provided.
datasentinel show token
Response
Copyright 2023 (c) Datasentinel- All rights reserved www.datasentinel.io
================================================================================
Token : <token value>
Organization : ds-data
Expiration Date : 2023-07-22 18:09:29

Proxy

Under certain circumstances, an intermediary proxy server is required to establish a connection between the agent and the platform server.

Set

  • create a json file with proxy settings and update the agent
JSON description
cat > proxy.json << EOF
{
"host": "myProxyHostName",
"port": 4587,
"user": "username (optional)",
"password": "value (optional)"
}
EOF
datasentinel set proxy -f proxy.json
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Proxy successfully set!
Proxy
host : myProxyHostName
port : 4587
user :
password :

Show

datasentinel show proxy
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Proxy
host : hostname
port : 4587
user :
password :

Delete

datasentinel delete proxy
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Proxy successfully deleted!
Proxy
host :
port : 0
user :
password :

Connections

Show

datasentinel show connections
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
name status state host port user
--------------------------------------------------------------------------------
:9342 enabled running pg-sales-1734 9342 datasentinel
tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Enable All

datasentinel enable all
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Connections enabled!
name status state host port user
--------------------------------------------------------------------------------
:9342 enabled running pg-sales-1734 9342 datasentinel
tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Disable All

datasentinel disable all
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Connections disabled!
name status state host port user
--------------------------------------------------------------------------------
:9342 disabled not running pg-sales-1734 9342 datasentinel
tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Connection

Add

  • Create a JSON file with connection settings
JSON
cat > newConnection.json << EOF
{
"host": "pg-sales-1734",
"port": 9342,
"user": "datasentinel",
"password": "sentinel",
"tags": "application=sales,environment=production,provider=amazon,datacenter=lyon",
}
EOF
datasentinel add connection myNewConnection -f newConnection.json
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Connection added!
Name : myNewConnection
Status : enabled
State : connected
Host : pg-sales-1734
Port : 9342
user : datasentinel
password : sentinel
Lock monitoring : on
Lock monitoring delay : 30s
Table monitoring : on
Query samples : off
Query monitoring : on
Optimize : on
Min calls : 2
Min time : 1s
tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Show

datasentinel show connection myNewConnection
  • Parameter
    The connection name
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Name : myNewConnection
Status : enabled
State : connected
Host : pg-sales-1734
Port : 9342
user : datasentinel
password : sentinel
Lock monitoring : on
Lock monitoring delay : 30s
Table monitoring : on
Query samples : off
Query monitoring : on
Optimize : on
Min calls : 2
Min time : 1s
tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Dump

Dump connection properties in JSON format
datasentinel dump connection myNewConnection
  • Parameter
    The connection name
Response
{
"name": "myNewConnection",
"host": "pg-sales-1734",
"port": 9342,
"user": "datasentinel",
"password": "sentinel",
"tags": "application=sales,environment=production,provider=amazon,datacenter=lyon ",
"enabled": true,
"connected": true,
"collection-rate": "low",
"samples": false,
"lock-monitoring": true,
"lock-monitoring-delay": 30,
"table-monitoring": true,
"query-monitoring": true,
"query-monitoring-optimize": true,
"query-monitoring-min-calls": 2,
"query-monitoring-min-time": 1
}

Delete

datasentinel delete connection myNewConnection
  • Parameter
    The connection name
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Connection successfully deleted!

Enable

Activate the connection and initiate connection attempts in order to start monitoring
datasentinel enable connection myNewConnection
  • Parameter
    The connection name
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Connection enabled!
Name : myNewConnection
Status : enabled
State : connected
Host : pg-sales-1734
Port : 9342
user : datasentinel
password : sentinel
Lock monitoring : on
Lock monitoring delay : 30s
Table monitoring : on
Query samples : off
Query monitoring : on
Optimize : on
Min calls : 2
Min time : 1s
tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Disable

datasentinel disable connection myNewConnection
  • Parameter
    The connection name
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Connection disabled!
Name : myNewConnection
Status : enabled
State : connected
Host : pg-sales-1734
Port : 9342
user : datasentinel
password : sentinel
Lock monitoring : on
Lock monitoring delay : 30s
Table monitoring : on
Query samples : off
Query monitoring : on
Optimize : on
Min calls : 2
Min time : 1s
tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Update

cat > updateConnection.json << EOF
{
"host": "pg-sales-1734",
"port": 9342,
"user": "datasentinel",
"password": "sentinel",
"tags": "application=sales,environment=production,provider=microsoft,datacenter=paris",
}
EOF
datasentinel update connection myNewConnection -f updateConnection.json
  • Parameter
    The connection name
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Connection updated!
Name : myNewConnection
Status : enabled
State : connected
Host : pg-sales-1734
Port : 9342
user : datasentinel
password : sentinel
Lock monitoring : on
Lock monitoring delay : 30s
Table monitoring : on
Query samples : off
Query monitoring : on
Optimize : on
Min calls : 2
Min time : 1s
tags : application=sales,environment=production,provider=amazon,datacenter=lyon

Collection Level

The Agent Version 3.2 introduces the Collection Level feature, which was released in February 2023 and allows you to choose the metrics to monitor.

Query Samples

datasentinel update connection ":9342" samples on|off
  • Parameter
    The connection name

Lock Monitoring

datasentinel update connection ":9342" lock-monitoring on|off
  • Parameter
    The connection name

Lock Monitoring Delay

datasentinel update connection ":9342" lock-monitoring-delay <seconds>
  • Parameters
    The connection name Seconds

Table Monitoring

datasentinel update connection ":9342" table-monitoring on|off
  • Parameter
    The connection name

Query Monitoring

datasentinel update connection ":9342" query-monitoring on|off
  • Parameter
    The connection name

Query Monitoring Optimize

datasentinel update connection ":9342" query-monitoring-optimize on|off
  • Parameter
    The connection name

Query Monitoring Min Calls

datasentinel update connection ":9342" query-monitoring-min-calls <value>
  • Parameters
    The connection name Minimum number of calls registered in pg_stat_statements

Query Monitoring Min Time

datasentinel update connection ":9342" query-monitoring-min-time <seconds>
  • Parameter
    The connection name Minimum total execution time (seconds)

Internals

Since Agent Version 3.2

Show Metrics

Displays internal metrics and caches for debugging purposes
datasentinel show metrics
JSON output
{
"version": "3.2.0",
"port": 8282,
"start_time": "2023-01-31 13:08:27",
"memory": "59MB",
"threads": 18,
"pg_connections": 2,
"running": 1,
"not_running": 1,
"connections": [
{
"name": "test",
"stat_statements": 0,
"stat_statements_cache": 0,
"stat_plans_cache": 0,
"query_ids_cache": 0,
"missing_query_ids": 0,
"plan_ids_cache": 0,
"cache_tables": 0,
"table_stats_cache": 0,
"index_stats_cache": 0,
"normal_task_runtime": 0,
"low_task_runtime": 0,
"tables_monitoring_limit": 2500,
"query_monitoring": true,
"query_monitoring_limit": 35000,
"query_monitoring_min_calls": 2,
"query_monitoring_min_time": 1,
"query_monitoring_optimize": true,
"xa_trans_optimized": 0,
"insert_values_optimized": 0,
"in_list_query_optimized": 0,
"execution_plan_optimized": 0,
"samples": false,
"lock_monitoring": true,
"lock_monitoring_delay": 10,
"pool_used": 0
},
{
"name": "test_connection",
"stat_statements": 109,
"stat_statements_cache": 109,
"stat_plans_cache": 52,
"query_ids_cache": 51,
"missing_query_ids": 0,
"plan_ids_cache": 48,
"cache_tables": 1,
"table_stats_cache": 148,
"index_stats_cache": 352,
"normal_task_runtime": 1663,
"low_task_runtime": 12069,
"tables_monitoring_limit": 2500,
"query_monitoring": true,
"query_monitoring_limit": 35000,
"query_monitoring_min_calls": 21,
"query_monitoring_min_time": 33,
"query_monitoring_optimize": true,
"xa_trans_optimized": 0,
"insert_values_optimized": 23,
"in_list_query_optimized": 1,
"execution_plan_optimized": 0,
"samples": true,
"lock_monitoring": true,
"lock_monitoring_delay": 10,
"pool_used": 3
}
]
}

Test Query Monitoring Optimization

The Query Optimize feature fetches queries from pg_stat_statements, filtered by minimum calls or minimum total execution time, groups similar queries, and shows the optimization gain at each level.
datasentinel update connection ":9342" test-query-monitoring-optimize
  • Parameter
    The connection name
JSON output
{
"pg_stat_statements": 964,
"min_calls_filter": 21,
"min_time_filter": "33s",
"filtered_statements_get_time": "153ms",
"filtered_statements_with_min_calls_or_min_time": 109,
"filtered_ratio": "89%",
"optimized_statements": 109,
"optimized_ratio": "0%",
"optimized_statements_run_time": "8ms"
}