agent systemd units
cback agents are normally deployed for production via systemd, we provide for this purpose some template systemd
unit files in files/units/
$ ll restic-backup-agent/files/units
total 28
-rw-r--r--. 1 root root 283 Mar 11 17:03 cback-backup@.service
-rw-r--r--. 1 root root 285 Mar 11 17:03 cback-enabler@.service
-rw-r--r--. 1 root root 281 Mar 11 17:03 cback-prune@.service
-rw-r--r--. 1 root root 285 Mar 11 17:03 cback-restore@.service
-rw-r--r--. 1 root root 283 Mar 11 17:03 cback-switch@.service
-rw-r--r--. 1 root root 279 Mar 11 17:03 cback-sync@.service
-rw-r--r--. 1 root root 283 Mar 11 17:03 cback-verify@.service
It is left to the reader to decide how to ensure these units are properly deployed on a target worker node in their infrastructure, if at all. If you wish to utilise these units quickly for testing, the bellow example demonstrates their usage:
# ensure systemd units are present in the correct location on the worker node
$ cp - restic-backup-agent/files/units/* /usr/lib/systemd/system/
# verify the units are in place with correct permissions
$ ll /usr/lib/systemd/system/ | grep cback
-rw-r--r--. 1 root root 283 May 17 10:53 cback-backup@.service
-rw-r--r--. 1 root root 285 May 17 10:53 cback-enabler@.service
-rw-r--r--. 1 root root 281 May 17 10:53 cback-prune@.service
-rw-r--r--. 1 root root 285 May 17 10:53 cback-restore@.service
-rw-r--r--. 1 root root 283 May 17 10:53 cback-switch@.service
-rw-r--r--. 1 root root 279 May 17 10:53 cback-sync@.service
-rw-r--r--. 1 root root 283 May 17 10:53 cback-verify@.service
# reload systemd units
$ systemctl daemon-reload
# enable a agent of choice
systemctl start cback-<type_of_agent>@<new_agent_id>
systemctl enable cback-<type_of_agent>@<new_agent_id>
# for example, enable a backup agent
$ systemctl start cback-backup@1
$ systemctl enable cback-backup@1
Created symlink /etc/systemd/system/multi-user.target.wants/cback-backup@1.service → /usr/lib/systemd/system/cback-backup@.service.
$ systemctl status cback-backup@1
● cback-backup@1.service - CERNBack Backup Agent 1
Loaded: loaded (/usr/lib/systemd/system/cback-backup@.service; enabled; vendor preset: disabled)
Active: active (running) since Fri 2024-05-17 10:55:56 CEST; 21s ago
Main PID: 1057351 (python)
Tasks: 1 (limit: 44457)
Memory: 62.4M
CGroup: /system.slice/system-cback\x2dbackup.slice/cback-backup@1.service
└─1057351 /usr/lib/python3.9/site-packages/cback/venv/bin/python /usr/bin/cback backup agent
May 17 10:55:56 devstack-cback-71abd95b7d.cern.ch cback[1057351]: 2024-05-17 10:55:56 DEBUG /usr/lib/python3.9/site-packages/cback/model/agents/agent.py:97 > agent is enabled pid=1057351 agent=backup id=0
May 17 10:55:56 devstack-cback-71abd95b7d.cern.ch cback[1057351]: 2024-05-17 10:55:56 DEBUG /usr/lib/python3.9/site-packages/cback/model/agents/agent.py:112 > agent will sleep for 4 secs pid=1057351 agent=backup id=0
May 17 10:56:00 devstack-cback-71abd95b7d.cern.ch cback[1057351]: 2024-05-17 10:56:00 DEBUG /usr/lib/python3.9/site-packages/cback/model/agents/agent.py:97 > agent is enabled pid=1057351 agent=backup id=0
May 17 10:56:01 devstack-cback-71abd95b7d.cern.ch cback[1057351]: 2024-05-17 10:56:01 DEBUG /usr/lib/python3.9/site-packages/cback/model/agents/agent.py:112 > agent will sleep for 3 secs pid=1057351 agent=backup id=0
May 17 10:56:04 devstack-cback-71abd95b7d.cern.ch cback[1057351]: 2024-05-17 10:56:04 DEBUG /usr/lib/python3.9/site-packages/cback/model/agents/agent.py:97 > agent is enabled pid=1057351 agent=backup id=0
May 17 10:56:04 devstack-cback-71abd95b7d.cern.ch cback[1057351]: 2024-05-17 10:56:04 DEBUG /usr/lib/python3.9/site-packages/cback/model/agents/agent.py:112 > agent will sleep for 1 secs pid=1057351 agent=backup id=0
May 17 10:56:05 devstack-cback-71abd95b7d.cern.ch cback[1057351]: 2024-05-17 10:56:05 DEBUG /usr/lib/python3.9/site-packages/cback/model/agents/agent.py:97 > agent is enabled pid=1057351 agent=backup id=0
May 17 10:56:05 devstack-cback-71abd95b7d.cern.ch cback[1057351]: 2024-05-17 10:56:05 DEBUG /usr/lib/python3.9/site-packages/cback/model/agents/agent.py:112 > agent will sleep for 9 secs pid=1057351 agent=backup id=0
May 17 10:56:14 devstack-cback-71abd95b7d.cern.ch cback[1057351]: 2024-05-17 10:56:14 DEBUG /usr/lib/python3.9/site-packages/cback/model/agents/agent.py:97 > agent is enabled pid=1057351 agent=backup id=0
May 17 10:56:14 devstack-cback-71abd95b7d.cern.ch cback[1057351]: 2024-05-17 10:56:14 DEBUG /usr/lib/python3.9/site-packages/cback/model/agents/agent.py:112 > agent will sleep for 7 secs pid=1057351 agent=backup id=0
[note] the above debug log from systemd / journalctl is useful, but somtimes its nice to check manually that the agent is actually running A.) as a cback process, B.) any actual restic operations (a job.) You can check as shown in the bellow example:
$ ps aux | grep -E 'restic|cback'
root 1057351 0.7 1.2 226164 87092 ? Ss 10:55 0:02 /usr/lib/python3.9/site-packages/cback/venv/bin/python /usr/bin/cback backup agent
root 1058019 0.0 0.9 734696 65612 pts/1 Sl+ 11:06 0:00 /usr/bin/restic --cache-dir /var/tmp/cback/devstack-70e1c17f-a565-4f48-9d26-c2c550c4f12f snapshots --path /cephfs-devstack/volumes/_nogroup/8dabc902-5d08-475f-8738-adef46510c60/f7139a00-1be9-4314-a6f3-7b95cde1070e --json
root 1057512 0.0 0.0 12144 1200 pts/0 R+ 11:00 0:00 grep --color=auto -E restic|cback