[https://lists.debian.org/debian-services-admin/2018/02/msg00000.html]

Hi,

We'd like to get some advise on the plans to move the KGB service provided for salsa-hosted projects (and others) to a debian-hosted machine.

https://wiki.debian.org/Services/KGB

Current architecture
====================

Three IRC bots, running on home servers. Client part is chained in post-commit hooks, running on the Git[1] server (alioth and others).

 [1] the client also works with Subversion and CVS (used by the web team)

When a push/commit/checkin is made, the client processes the change and sends one of the bots a notification to be relayed on IRC. If there is a problem with a given bot, it retries with another, until the list is over, at which point a warning is issued.

Each repository needs to be configured on the bot side, so that requests can be authenticated. Every repository is given access to a fixed number of IRC channels. All requests are authenticated using a hash over the request content and a pre-shared secret.

Apart from post-commit hooks, there is a separate functionality which allows sending free-text messages to IRC, suitable for purposes other than commit notification - bug submission, monitoring alert, job completion etc.

This approach has two downsides:

 - that every new project needs to be configured by bot admins (e.g. three times)
 
 - Sometimes bots are inaccessible (e.g. power outage or hardware/link problems). This is somewhat mitigated by the client retrying with a different bot, but there is still a delay.

The future
==========

Most of the projects move to salsa, which doesn't support post-commit hooks, but can use webhooks.

For these, a webhook termination is implemented in the bot directly. The webhook termination code doesn't need authentication, because it trusts salsa's IP address. The variable parts of the notification (e.g. channel/network, use of colors etc) can be embedded in the webhook URI. We trust the projects not to abuse the system by broadcasting advertisments to thousands of IRC channels (because they obey DMUP).

On service maintainers' side, this avoids the need to configure individual salsa projects. For users it is also much easier to enable IRC notifications - instead of fiddling with post-commit hooks and their configuration they enable the webhook support in gitlab as described in https://salsa.debian.org/kgb-team/kgb/wikis/usage

The "old" scheme with authenticated projects stays, because it would be used by the non-gitlab projects and the projects that need free-text notifications.

To address both the stability issue with home servers and the need to configure several servers, we'd like to move the service to a single debian-hosted machine.

Hosting requirements
====================

The expected load is rather low, CPU% is seldom higher than 0.0. Memory footprint was seen to be 200M after 30 days of running, and the expected network bandwidth is about xxx MB/month.

In terms of access, we would want to be able to:

 - install/upgrade kgb-bot and kgb-client packages (plus dependencies), possibly from backports or unstable (or from git)
 
 - start/stop/restart/reload/status the kgb-bot service
 
 - read/modify KGB's configuration:
 
   - modify files under /etc/kgb-bot/*
   
   - create/modify/delete files under /etc/kgb-bot/kgb.d/
   
   currently permissions look like
   
   drwxr-xr-x root root        /etc/kgb-bot
   -rw-r----- root Debian-kgb  kgb.conf
   drwxr-x--- root Debian-kgb  kgb.conf.d
   -rw-r----- root Debian-kgb  kgb.conf.dpkg-dist
   -rw-r----- root Debian-kgb  3dprinter.conf
   -rw-r----- root Debian-kgb  android-tools.conf
   -rw-r----- root Debian-kgb  apt.conf
   -rw-r----- root Debian-kgb  archlinux-reproducible.conf
   -rw-r----- root Debian-kgb  boinc.conf
   .....
   
 - read /var/log/kgb-bot.log* (-rw-r--r-- Debian-kgb:root)


For the KGB service admins,
	Damyan Ivanov