a34c38a4

By: ananjiani <ananjiani@noreply.codeberg.org>

fix(k3s): fail closed on Marinara writer-lease drain (#304)

## Summary

- give Marinara 120 seconds to terminate, signal its PID 1 wrapper from `preStop`, and keep the hook active until the writer lease disappears and PID 1 exits
- add a one-replica PodDisruptionBudget so unattended drains cannot evict Marinara's only writer
- run the shared k3s drain before Comin requests reboot, honor PDBs, use a 180-second drain timeout, and propagate failure
- preserve an existing operator cordon and uncordon only when the failed preflight created the cordon
- allow systemd 240 seconds for the drain and cleanup paths

## Safety behavior

A k3s node that currently hosts Marinara cannot complete its unattended pre-reboot drain. Comin skips the reboot, and the node returns to its prior scheduling state. The hook never deletes `.writer-lease`; Marinara's normal close path remains responsible for lease removal.

This PR does not change upstream Marinara code.

## Validation

- `nix develop --command pre-commit run --files <changed files>`
- targeted Nix evaluations for `samwise`, `theoden`, `aragorn`, and `rivendell`
- generated `comin-auto-reboot` and `k3s-drain` script inspection and shell syntax checks
- generated drain failure-path tests for success, PDB rejection, existing cordon, and uncordon failure
- `kubectl kustomize k8s/apps/marinara`
- rendered Deployment and PDB assertions
- server-side dry run against the live Kubernetes API
- `git diff --check`
- added-line secret and injection scan

## Deferred disruptive checks

After merge and GitOps deployment, run one controlled Marinara pod termination test. Move Marinara deliberately before any manual reboot of its host. Do not test a live node reboot until the pod shutdown test proves that `.writer-lease` disappears without manual removal.

Co-authored-by: Ammar Nanjiani <ammar.nanjiani@gmail.com>
Reviewed-on: https://codeberg.org/ananjiani/infra/pulls/304

Total

1m49s

Timeline