You could write a script that just restarts your container, make sure unprivileged users cannot edit it, and do one of two things:

1. make a sudoers entry for your unprivileged account to call just that script as a user in the docker group with sudo
2. use setuid on the script to have it execute from the docker group even when executed by users

K8s has a mild solution to chicken and egg situations for nodes - the nodes support ‘static manifests’ which can be pods they know how to bring up before ever connecting to the API server. So you could have your wireguard peer be brought up this way. Downside is while those static manifests show up in k8s APIs, they aren’t fully manageable since they are defined by files on disk.

Sometimes I wonder if in 75 years people will look back on our caffeine use in this generation like we currently look back at cocaine use in products in the 19th century. Until then, I continue to slurp down coffee like that is my actual job.

Yea it’s very easy to learn enough to run, it has built-in service discovery and secrets now, and writing parameterized jobs feels so much nicer than a helm chart in k8s.

10/10, would orchestrate again

I use k8s at work a lot - I choose to use Nomad at home, you may want to add that to your shortlist.

I am nearly complete migrating my ceph cluster and nomad compute cluster to arm :shrug:

My day job is a lot of kube/openshift so nomad is refreshing. Having the template blocks are amazing and makes it so that much of what helm gave me is not required. Parameterized jobs are the best once you find a good use case for them!

• 33 nomad jobs, most being containers
• 12 physical nomad clients
  • 3 amd64 poweredge
  • 2 pi4
  • 6 Nano Pi r5c
  • 1 odroid M1
• Ceph: (nomad orchestrated)
  • 8 OSD
  • 50TB total raw disk

Every time we see this in our legacy code we yell out: dolla-dolla bills 'yall!

(after) …ah crap it’s actually selinux…

Nomad is a breath of fresh air after working with k8s professionally.

Don’t get me wrong, love k8s, but it’s a bit much (until you need it)

Ceph is excellent as a distributed storage solution - but should really have 4 machines with 2 or more drives each to reach a good level of redundancy - which is a bit much for most people on this sub.

One nice feature is it deals with heterogeneous drives well, like if you need to buy a bunch of used ones on eBay for cheap.

Probably not a good solution for your case because of the footprint - but good to be aware of it.

Ok that’s pretty cool (good bot)

How excellent for my MQTT behind nginx next to my pihole

:)

Nanopi R5C

Nanopi R5C. Cute little buggers too.

I have 2 pi4 4GB boards and was waiting forever to get a third to run RAFT based services across.

I gave up last year and bought 3 chinese boards at $60/ea with 2x 2.5Gb Ethernet each, emmc, and a m.2 slot - and they run at half the temp of the pi4 boards.

I never needed the wifi/bt and form-factor the pi boards offered anyway - really no reason to stay as long as you can find software that boots on other boards.

Ceph (rbd,s3) on 4 poweredges.

Nomad, Consul servers running in a 3 node raft on some ARM SBCs.

Nomad clients on 2 poweredges and 3 arm SBCs running:

• s3 CSI (media and large file storage is all on s3)
• rbd CSI (all the stateful jobs have rbd block devices)
• NATS cluster with Jetstream to enable MQTT support
• mosquito mqtt (had to run mosquito for mqtt v5)
• mosquito<->nats bridge (via benthos)
• nodered (just for easy Google home integration)
• zwavejs2mqtt
• zigbee2mqtt
• frigate (can only talk mqtt v5 so had to run mosquito just for this)
• grafana
• gotify
• gitea
• drone CI
• postgresql
• BitTorrent client
• ceph rgw s3 gateways
• NATS based home automation lambdas - I wrote these in go
• Adguard home
• traefik as main ingress
• Prometheus
• prom node exporter
• jellyfin
• jackett
• a program I wrote that manages torznab->acquisiton->s3 lifecycle
• a website
• wireguard servers

And that is just in the server room - I also have more like the 3d printer and CNC machine controllers etc.