CLI
The agent offers a CLI for conveniently modifying attributes and connections.
The agent stores its configuration in the hidden .datasentinel directory within the user's home directory, using two files: agent.yml and connections.yml. These files can be manually managed if necessary.
Prerequisites
Enable the execution of the CLI
Using an alias
alias datasentinel='DATASENTINEL_PATH=<<INSTALL_DIRECTORY>>/datasentinel;LD_LIBRARY_PATH=$DATASENTINEL_PATH/lib/ $DATASENTINEL_PATH/datasentinel'Using environment variables
export DATASENTINEL_PATH="<<INSTALL_DIRECTORY>>/datasentinel"
export LD_LIBRARY_PATH=$DATASENTINEL_PATH/lib:$LD_LIBRARY_PATH
export PATH=$DATASENTINEL_PATH:$PATHAgent Management
Show Commands
Simply enter the datasentinel command.
datasentinelResponse
Copyright 2023 (c) Datasentinel- All rights reserved www.datasentinel.io
================================================================================
Agent:
- start agent
- stop agent
- status agent
- show metrics
Show internal agent metrics and cache activity
- set port <port number>
- set collection-rate <low|high>
low: The sessions collection is done every 10 seconds
high: The sessions collection is done every second (Default value)
If the datasentinel extension is not installed, the collection-rate is automatically adjusted to low value
- set tables-monitoring-limit (default 1000)
The agent monitors the activity of tables and indexes if the number of tables in the connection is less than the defined limit
- set query-monitoring-limit (default 10000)
The agent tracks query activity, storing unique IDs in a daily cache. Exceeding limit triggers automatic deactivation of query monitoring.
Connections: when the connections are disabled, the agent is disconnected.
- enable all
- disable all
Connection:
- add connection <name> -f <json file>
- update connection <name> -f <json file>
json example: {
"host": "hostname",
"port": 4587,
"user": "username",
"password": "value",
"tags": "key=value,key=value,..."
}
- update connection <name> samples <on|off> (default off)
Collect and send sample queries, with literal values if present
- update connection <name> lock-monitoring <on|off> (default on)
Scan blocking sessions (longer than lock-monitoring-delay) and send blocking reports
- update connection <name> lock-monitoring-delay <seconds> (default 30)
Minimum number of seconds before triggering a blocking scenario
- update connection <name> table-monitoring <on|off> (default on)
Monitor the activity of tables and indexes (The number of tables must be lower than the maximum limit set at the agent level (default 1000)
- update connection <name> query-monitoring <on|off> (default on)
Monitor pg_stat_statements query activity
- update connection <name> query-monitoring-optimize <on|off> (default on)
Group similar pg_stat_statements queries
- update connection <name> query-monitoring-min-calls <calls> (default 2)
- update connection <name> query-monitoring-min-time <seconds> (default 1)
Retrieve pg_stat_statements queries executed n times (calls) or with total execution time exceeding specified seconds
- update connection <name> test-query-monitoring-optimize
Test and optimize pg_stat_statements query contents with previous parameters, display results
- delete connection <name>
- enable connection <name>
- disable connection <name>
- show connections
- show connection <name>
Dump connection in JSON format:
- dump connection <name>
Upload server:
- set server <host> <port>
- show server
- test server
Token:
- set token <value>
- show token
Proxy:
- set proxy -f <json file>
json example: {
"host": "hostname",
"port": 4587,
"user": "username (optional)",
"password": "value (optional)"
}
- delete proxy
- show proxyStatus
datasentinel status agentResponse
Copyright 2023 (c) Datasentinel- All rights reserved www.datasentinel.io
================================================================================
Agent
Version : 3.2.0
Server : pgsentinel.localdomain
Port : 8282
Start time : 2023-01-31 13:08:27
Collection rate : high
Table monitoring limit : 2500
Query monitoring limit : 35000
Sql max size : 7512453
Proxy
host :
port : 0
user :
password :
Upload
host : 163.172.135.154
port : 443
Connections
declared : 2
running : 1
not running : 1Start
datasentinel start agentResponse
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Starting the agent...
To show the status : datasentinel status agentStop
datasentinel stop agentResponse
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
The agent is stopped!Change Port Number
By default, the agent is configured to listen on port 8282, enabling API or CLI calls to be made.
datasentinel set port 9121Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Agent port successfully set! The agent has been stopped, you need to restart itChange Table Monitoring Limit
datasentinel set tables-monitoring-limit <limit>Parameter:
The limit number. The agent tracks the activity of tables and indexes if the connection’s table count is below the set limit (default 1000).
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Tables monitoring limit successfully set to 3000!
Agent
Version : 3.2.0
Server : pgsentinel.localdomain
Port : 8282
Start time : 2023-01-31 13:08:27
Collection rate : low
Table monitoring limit : 2500
Query monitoring limit : 35000
Sql max size : 7512453
Proxy
host :
port : 0
user :
password :
Upload
host : 163.172.135.154
port : 443
Connections
declared : 2
running : 1
not running : 1Change Query monitoring Limit
Since agent version 3.2
datasentinel set query-monitoring-limit <limit>Parameter:
The limit number. The agent tracks query activity, storing unique IDs in a daily cache. Exceeding limit triggers automatic deactivation of query monitoring. (default 30000)
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Tables monitoring limit successfully set to 3000!
Agent
Version : 3.2.0
Server : pgsentinel.localdomain
Port : 8282
Start time : 2023-01-31 13:08:27
Collection rate : low
Table monitoring limit : 2500
Query monitoring limit : 35000
Sql max size : 7512453
Proxy
host :
port : 0
user :
password :
Upload
host : 163.172.135.154
port : 443
Connections
declared : 2
running : 1
not running : 1Upload Server
The Upload Server refers to the server name of the platform where metrics are transmitted.
Set
Change the server where metrics are sent
datasentinel set server app.datasentinel.io 443Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Server successfully set!
Server
host : app.datasentinel.io
port : 443Show
Show the platform server where metrics are sent
datasentinel show serverResponse
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Server
host : app.datasentinel.io
port : 443Test
datasentinel test serverResponse
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
TEST: The upload server is reachable and up!
Server
host : app.datasentinel.io
port : 443Token
Set
Configure the authentication token used to access the platform. This should only be done during initialization. The agent subsequently retrieves updated tokens from the platform.
datasentinel set token <token value>Response
Copyright 2023 (c) Datasentinel- All rights reserved www.datasentinel.io
================================================================================
Token successfully set!
Server
host : app.datasentinel.io
port : 443Show
After testing the token on the platform's upload server, the validity date of the token is provided.
datasentinel show tokenResponse
Copyright 2023 (c) Datasentinel- All rights reserved www.datasentinel.io
================================================================================
Token : <token value>
Organization : ds-data
Expiration Date : 2023-07-22 18:09:29Proxy
Under certain circumstances, an intermediary proxy server is required to establish a connection between the agent and the platform server.
Set
create a json file with proxy settings and update the agent
JSON description
cat > proxy.json << EOF
{
"host": "myProxyHostName",
"port": 4587,
"user": "username (optional)",
"password": "value (optional)"
}
EOFdatasentinel set proxy -f proxy.jsonResponse
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Proxy successfully set!
Proxy
host : myProxyHostName
port : 4587
user :
password :Show
datasentinel show proxyResponse
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Proxy
host : hostname
port : 4587
user :
password :Delete
datasentinel delete proxyResponse
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Proxy successfully deleted!
Proxy
host :
port : 0
user :
password :Connections
Show
datasentinel show connectionsResponse
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
name status state host port user
--------------------------------------------------------------------------------
:9342 enabled running pg-sales-1734 9342 datasentinel
tags : application=sales,environment=production,provider=amazon,datacenter=lyonEnable All
datasentinel enable allResponse
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Connections enabled!
name status state host port user
--------------------------------------------------------------------------------
:9342 enabled running pg-sales-1734 9342 datasentinel
tags : application=sales,environment=production,provider=amazon,datacenter=lyonDisable All
datasentinel disable allResponse
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Connections disabled!
name status state host port user
--------------------------------------------------------------------------------
:9342 disabled not running pg-sales-1734 9342 datasentinel
tags : application=sales,environment=production,provider=amazon,datacenter=lyonConnection
Add
Create a JSON file with connection settings
JSON
cat > newConnection.json << EOF
{
"host": "pg-sales-1734",
"port": 9342,
"user": "datasentinel",
"password": "sentinel",
"tags": "application=sales,environment=production,provider=amazon,datacenter=lyon",
}
EOFdatasentinel add connection myNewConnection -f newConnection.jsonResponse
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Connection added!
Name : myNewConnection
Status : enabled
State : connected
Host : pg-sales-1734
Port : 9342
user : datasentinel
password : sentinel
Lock monitoring : on
Lock monitoring delay : 30s
Table monitoring : on
Query samples : off
Query monitoring : on
Optimize : on
Min calls : 2
Min time : 1s
tags : application=sales,environment=production,provider=amazon,datacenter=lyonShow
datasentinel show connection myNewConnectionParameter
The connection name
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Name : myNewConnection
Status : enabled
State : connected
Host : pg-sales-1734
Port : 9342
user : datasentinel
password : sentinel
Lock monitoring : on
Lock monitoring delay : 30s
Table monitoring : on
Query samples : off
Query monitoring : on
Optimize : on
Min calls : 2
Min time : 1s
tags : application=sales,environment=production,provider=amazon,datacenter=lyonDump
Dump connection properties in JSON format
datasentinel dump connection myNewConnectionParameter
The connection name
Response
{
"name": "myNewConnection",
"host": "pg-sales-1734",
"port": 9342,
"user": "datasentinel",
"password": "sentinel",
"tags": "application=sales,environment=production,provider=amazon,datacenter=lyon ",
"enabled": true,
"connected": true,
"collection-rate": "low",
"samples": false,
"lock-monitoring": true,
"lock-monitoring-delay": 30,
"table-monitoring": true,
"query-monitoring": true,
"query-monitoring-optimize": true,
"query-monitoring-min-calls": 2,
"query-monitoring-min-time": 1
}Delete
datasentinel delete connection myNewConnectionParameter
The connection name
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Connection successfully deleted!Enable
Activate the connection and initiate connection attempts in order to start monitoring
datasentinel enable connection myNewConnectionParameter
The connection name
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Connection enabled!
Name : myNewConnection
Status : enabled
State : connected
Host : pg-sales-1734
Port : 9342
user : datasentinel
password : sentinel
Lock monitoring : on
Lock monitoring delay : 30s
Table monitoring : on
Query samples : off
Query monitoring : on
Optimize : on
Min calls : 2
Min time : 1s
tags : application=sales,environment=production,provider=amazon,datacenter=lyonDisable
datasentinel disable connection myNewConnectionParameter
The connection name
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Connection disabled!
Name : myNewConnection
Status : enabled
State : connected
Host : pg-sales-1734
Port : 9342
user : datasentinel
password : sentinel
Lock monitoring : on
Lock monitoring delay : 30s
Table monitoring : on
Query samples : off
Query monitoring : on
Optimize : on
Min calls : 2
Min time : 1s
tags : application=sales,environment=production,provider=amazon,datacenter=lyonUpdate
cat > updateConnection.json << EOF
{
"host": "pg-sales-1734",
"port": 9342,
"user": "datasentinel",
"password": "sentinel",
"tags": "application=sales,environment=production,provider=microsoft,datacenter=paris",
}
EOFdatasentinel update connection myNewConnection -f updateConnection.jsonParameter
The connection name
Response
Copyright 2023 (c) datasentinel- All rights reserved www.datasentinel.io
================================================================================
Connection updated!
Name : myNewConnection
Status : enabled
State : connected
Host : pg-sales-1734
Port : 9342
user : datasentinel
password : sentinel
Lock monitoring : on
Lock monitoring delay : 30s
Table monitoring : on
Query samples : off
Query monitoring : on
Optimize : on
Min calls : 2
Min time : 1s
tags : application=sales,environment=production,provider=amazon,datacenter=lyonCollection Level
Query Samples
datasentinel update connection ":9342" samples on|offParameter
The connection name
Lock Monitoring
datasentinel update connection ":9342" lock-monitoring on|offParameter
The connection name
Lock Monitoring Delay
datasentinel update connection ":9342" lock-monitoring-delay <seconds>Parameters
The connection name Seconds
Table Monitoring
datasentinel update connection ":9342" table-monitoring on|offParameter
The connection name
Query Monitoring
datasentinel update connection ":9342" query-monitoring on|offParameter
The connection name
Query Monitoring Optimize
datasentinel update connection ":9342" query-monitoring-optimize on|offParameter
The connection name
Query Monitoring Min Calls
datasentinel update connection ":9342" query-monitoring-min-calls <value>Parameters
The connection name Minimum number of calls registered in pg_stat_statements
Query Monitoring Min Time
datasentinel update connection ":9342" query-monitoring-min-time <seconds>Parameter
The connection name Minimum total execution time (seconds)
Internals
Since Agent Version 3.2
Show Metrics
Displays internal metrics and caches for debugging purposes
datasentinel show metricsJSON output
{
"version": "3.2.0",
"port": 8282,
"start_time": "2023-01-31 13:08:27",
"memory": "59MB",
"threads": 18,
"pg_connections": 2,
"running": 1,
"not_running": 1,
"connections": [
{
"name": "test",
"stat_statements": 0,
"stat_statements_cache": 0,
"stat_plans_cache": 0,
"query_ids_cache": 0,
"missing_query_ids": 0,
"plan_ids_cache": 0,
"cache_tables": 0,
"table_stats_cache": 0,
"index_stats_cache": 0,
"normal_task_runtime": 0,
"low_task_runtime": 0,
"tables_monitoring_limit": 2500,
"query_monitoring": true,
"query_monitoring_limit": 35000,
"query_monitoring_min_calls": 2,
"query_monitoring_min_time": 1,
"query_monitoring_optimize": true,
"xa_trans_optimized": 0,
"insert_values_optimized": 0,
"in_list_query_optimized": 0,
"execution_plan_optimized": 0,
"samples": false,
"lock_monitoring": true,
"lock_monitoring_delay": 10,
"pool_used": 0
},
{
"name": "test_connection",
"stat_statements": 109,
"stat_statements_cache": 109,
"stat_plans_cache": 52,
"query_ids_cache": 51,
"missing_query_ids": 0,
"plan_ids_cache": 48,
"cache_tables": 1,
"table_stats_cache": 148,
"index_stats_cache": 352,
"normal_task_runtime": 1663,
"low_task_runtime": 12069,
"tables_monitoring_limit": 2500,
"query_monitoring": true,
"query_monitoring_limit": 35000,
"query_monitoring_min_calls": 21,
"query_monitoring_min_time": 33,
"query_monitoring_optimize": true,
"xa_trans_optimized": 0,
"insert_values_optimized": 23,
"in_list_query_optimized": 1,
"execution_plan_optimized": 0,
"samples": true,
"lock_monitoring": true,
"lock_monitoring_delay": 10,
"pool_used": 3
}
]
}Test Query Monitoring Optimization
The Query Optimize feature fetches queries from pg_stat_statements, filtered by minimum calls or minimum total execution time, groups similar queries, and shows the optimization gain at each level.
datasentinel update connection ":9342" test-query-monitoring-optimizeParameter
The connection name
JSON output
{
"pg_stat_statements": 964,
"min_calls_filter": 21,
"min_time_filter": "33s",
"filtered_statements_get_time": "153ms",
"filtered_statements_with_min_calls_or_min_time": 109,
"filtered_ratio": "89%",
"optimized_statements": 109,
"optimized_ratio": "0%",
"optimized_statements_run_time": "8ms"
}Last updated