Unexpected Agent Termination

In some cases, the Agent may stop running without any visible error or warning.

When the Agent is started from an interactive user session (for example via SSH), it is attached to the systemd user session scope created for that login, even if the process is forked.

Even if the Agent continues running for hours, it remains associated with that session.

Inspecting the Agent’s control group while it is running shows that it belongs to a session scope:

$ cat /proc/<<AGENT_PID>>/cgroup
/user.slice/user-0.slice/session-38188.scope

This confirms that the Agent is bound to a user session and not managed as a system service.

What Happens at Logout

When the user session ends, systemd-logind eventually cleans up the session and its associated resources. This cleanup includes terminating all processes running inside the session scope.

This behavior can occur:

• Immediately at logout, or
• Delayed, sometimes hours later, depending on system policies and resource cleanup timing.

Log Analysis

The Agent log file does not contain any error or warning messages prior to the shutdown. The last recorded log entries show that the Agent was running normally and actively sending data:

tail -2 ~/datasentinel/log/datasentinel.log

2026-01-24 08:31:06 - datasentinel - INFO - pg status sent
2026-01-24 08:31:10 - datasentinel - INFO - Alerting payload sent

Shortly afterward, systemd-logind closes the user session:

journalctl --since "2026-01-24 08:31:00" --until "2026-01-24 08:32:00"

Jan 24 08:31:21 systemd[1]: session-38188.scope: Deactivated successfully.
Jan 24 08:31:21 systemd-logind: Removed session 38188.

Root Cause

• The Agent was started manually from a login session
• It was placed in a systemd session scope
• When the session was cleaned up, systemd terminated the scope
• The Agent was killed as part of this cleanup

This is expected Linux behavior.

Recommended Solution

To prevent unexpected termination

• Run the Agent as a systemd service

This guarantees the Agent remains running independently of user logins or logouts.

This example below requires Agent version 3.9 or later, as the DATASENTINEL_FOREGROUND environment variable was introduced in this version. Create a new systemd service file at /usr/lib/systemd/system/datasentinel-agent.service

## Basic systemd Service
[Unit]
Description=DataSentinel Agent
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=postgres

Environment=DATASENTINEL_FOREGROUND=1

ExecStart=/var/lib/pgsql/datasentinel/datasentinel start agent
ExecStop=/var/lib/pgsql/datasentinel/datasentinel stop agent

# Restart automatically if it crashes
#Restart=on-failure

[Install]
WantedBy=multi-user.target

Enable and start the service

systemctl daemon-reload
systemctl enable datasentinel-agent
systemctl start datasentinel-agent
systemctl status datasentinel-agent

● datasentinel-agent.service - DataSentinel Agent
     Loaded: loaded (/usr/lib/systemd/system/datasentinel-agent.service; enabled; preset: disabled)
     Active: active (running) since Fri 2026-01-30 13:23:45 CET; 1s ago
   Main PID: 76505 (datasentinel)
      Tasks: 7 (limit: 48683)
     Memory: 51.0M
        CPU: 1.250s
     CGroup: /system.slice/datasentinel-agent.service
             └─76505 /var/lib/pgsql/datasentinel/datasentinel start agent

Jan 30 13:23:45 pg-crm-4743 systemd[1]: Started DataSentinel Agent.