sssd rides core: LDAP-sourced identity fleet-wide, additive alongside local lament #119

Merged
lament merged 6 commits from sssd into main 2026-07-31 00:19:30 -05:00
Owner
  • sssd (id_provider=ldap, auth_provider=krb5) rides core - every host resolves lament's identity from verdandi's LDAP directory, auth still goes through Kerberos directly rather than an LDAP simple-bind
    • ldap_id_use_start_tls = false - openldap.nix deliberately serves no TLS over internal (WG is the trust boundary), sssd defaults to StartTLS and fails hard against a server that never offers it
    • bind credential reuses the existing svc-ldap search account; environmentFile needs KEY=value shape but the sops secret is a bare password, so sops.templates wraps it rather than pointing environmentFile straight at the raw secret
    • cache_credentials/krb5_store_password_if_offline - verified live: pulled minerva's WG interface and logged in through the Proxmox console on the cached ticket
    • purely additive - local lamentUser accounts are untouched, this is proving the mechanism before any host's local account goes away
  • .sops.yaml/sops/identity.yaml: ldapBindPassword's key group widens to all five hosts now that sssd needs it fleet-wide, not just minerva/athena/verdandi
  • CI: homeConfigurations get their own build step, not just evaluated by flake check - lament@ishtar was never actually build-checked before this
  • ishtar's standalone HM config renames lament@ishtar -> lament@desktop (no behavior change, just a name that isn't a lie if a second standalone profile ever exists)

tried and reverted this session, worth knowing about even though the diff doesn't show it:

  • a sssd-authenticated NFSv4 (sec=krb5p) shared home dir across the fate hosts + athena, plus splitting those hosts onto standalone home-manager to match. both abandoned - the NFS mount hit a persistent ESTALE that survived every fix tried (fsid=0 pseudo-root removal, dropping nconnect, pinning SHA1 Kerberos enctypes around a real kernel GSS bug) and clean reboots of client/server/KDC. systemd-homed was the fallback idea; ruled out too - it wants to own the user record and its own unlock secret, which fights sssd/LDAP identity, Kerberos-ticket SSH, and standalone HM all at once
  • removing the local lament account entirely and going LDAP-only. blocked hard on two fronts: home-manager's integrated NixOS module reads config.users.users.<name>.name/.home directly at eval time with no override-safe escape hatch on the pinned rev (nix-community/home-manager#5244, unmerged fix in #6640), and NSS resolution order (files then sss) means a local account - even a password-less eval-only stub - still wins name-based lookups over LDAP, so sudo/group resolution needs either a local stub kept in sync at the right gidNumbers or a fleet-wide NSS reorder, neither of which got proven safe tonight. broke sudo live on verdandi, needed two generation-rollback reboots to recover. local lamentUser (password, SSH keys, extraGroups) is unchanged

left for later: the actual local-account removal needs its own low-stakes test pass (not a live host) before touching real access control again - NSS ordering, group GID matching, and the home-manager eval-time coupling all need to be solid first.

- `sssd` (`id_provider=ldap`, `auth_provider=krb5`) rides `core` - every host resolves `lament`'s identity from verdandi's LDAP directory, auth still goes through Kerberos directly rather than an LDAP simple-bind - `ldap_id_use_start_tls = false` - `openldap.nix` deliberately serves no TLS over `internal` (WG is the trust boundary), sssd defaults to StartTLS and fails hard against a server that never offers it - bind credential reuses the existing `svc-ldap` search account; `environmentFile` needs `KEY=value` shape but the sops secret is a bare password, so `sops.templates` wraps it rather than pointing `environmentFile` straight at the raw secret - `cache_credentials`/`krb5_store_password_if_offline` - verified live: pulled minerva's WG interface and logged in through the Proxmox console on the cached ticket - purely additive - local `lamentUser` accounts are untouched, this is proving the mechanism before any host's local account goes away - `.sops.yaml`/`sops/identity.yaml`: `ldapBindPassword`'s key group widens to all five hosts now that `sssd` needs it fleet-wide, not just minerva/athena/verdandi - CI: `homeConfigurations` get their own build step, not just evaluated by `flake check` - `lament@ishtar` was never actually build-checked before this - ishtar's standalone HM config renames `lament@ishtar` -> `lament@desktop` (no behavior change, just a name that isn't a lie if a second standalone profile ever exists) tried and reverted this session, worth knowing about even though the diff doesn't show it: - a `sssd`-authenticated NFSv4 (`sec=krb5p`) shared home dir across the fate hosts + athena, plus splitting those hosts onto standalone home-manager to match. both abandoned - the NFS mount hit a persistent ESTALE that survived every fix tried (`fsid=0` pseudo-root removal, dropping `nconnect`, pinning SHA1 Kerberos enctypes around a real kernel GSS bug) and clean reboots of client/server/KDC. `systemd-homed` was the fallback idea; ruled out too - it wants to own the user record and its own unlock secret, which fights `sssd`/LDAP identity, Kerberos-ticket SSH, and standalone HM all at once - removing the local `lament` account entirely and going LDAP-only. blocked hard on two fronts: home-manager's integrated NixOS module reads `config.users.users.<name>.name`/`.home` directly at eval time with no override-safe escape hatch on the pinned rev (`nix-community/home-manager#5244`, unmerged fix in `#6640`), and NSS resolution order (`files` then `sss`) means a local account - even a password-less eval-only stub - still wins name-based lookups over LDAP, so sudo/group resolution needs either a local stub kept in sync at the right `gidNumber`s or a fleet-wide NSS reorder, neither of which got proven safe tonight. broke `sudo` live on `verdandi`, needed two generation-rollback reboots to recover. local `lamentUser` (password, SSH keys, `extraGroups`) is unchanged left for later: the actual local-account removal needs its own low-stakes test pass (not a live host) before touching real access control again - NSS ordering, group GID matching, and the home-manager eval-time coupling all need to be solid first.
- nfs-server (minerva) + nfs-client (brigid/verdandi/athena): NFSv4.2 export of
  /srv/nfs-export, sec=krb5p riding the realm every host already has. add-tab
  justfile recipe mints a second principal (nfs/minerva) into an existing
  keytab instead of overwriting it.
- lament@server: generic standalone home-manager profile for the whole
  non-desktop fleet (minerva/brigid/verdandi/athena), same shape as ishtar's
  now-renamed lament@desktop. lament.standalone = true on all four.
- CI: build homeConfigurations as their own step, not just evaluated by flake
  check.
- .sops.yaml/sops/identity.yaml: widened to brigid+ishtar now that sssd rides
  core fleet-wide.
nfs-server/nfs-client removed, host.nix imports reverted, krb5 enctype pin
backed out. multiple root causes chased and partially fixed (fsid=0 pseudo-root
anti-pattern, nconnect+krb5p GSS context churn) but a persistent ESTALE
survived clean reboots of client/server/KDC even with all known fixes applied
- wire capture showed clean RPC replies, so the failure is happening somewhere
client-side that two research passes couldn't pin down. falling back to
systemd-homed for the roaming-home goal instead.

lament@server/lament@desktop standalone HM split stays - independent of NFS.
only existed to serve the shared-home effort (NFS, then systemd-homed), both
abandoned tonight. minerva/brigid/verdandi/athena go back to integrated HM
(home-manager.users.lament riding nixos-rebuild), lament@server config
removed. ishtar's lament@desktop split is untouched - unrelated, predates this.
users.users.lament drops hashedPassword/authorizedKeys/extraGroups entirely -
it exists purely because home-manager's integrated NixOS module reads
config.users.users.<name>.name/.home directly at eval time with no override-
safe escape hatch on the pinned rev (nix-community/home-manager#5244, #6640
unmerged). uid stays 1000 to match LDAP's - a mismatched uid would split
ownership between the local entry and anything LDAP/sssd touches, confirmed
via NSS getpwnam/getpwuid research before picking this.

openldap.nix: two new posixGroups (cn=wheel, cn=systemd-journal, both
memberUid: lament) so sssd's NSS group resolution can actually grant sudo/
journal access from the directory - cn=admins (groupOfNames, for forgejo's
OIDC role mapping) can't double as this, memberOf group membership isn't
memberUid-readable.

known gap, not closed tonight: NSS order is files-then-sss, so a name-based
lookup (login/sudo/id) still resolves via this local entry rather than LDAP -
only closed by reordering NSS fleet-wide, deliberately punted as its own
decision. what's real tonight: nothing local can authenticate on its own
anymore.

tested on minerva first, deploying fleet-wide next.
revert local lament account removal: broke sudo access live on verdandi
All checks were successful
CI / check (pull_request) Successful in 2m8s
CI / deploy (pull_request) Has been skipped
CI / build (pull_request) Successful in 4m50s
CI / dry-activate (pull_request) Successful in 37s
CI / check (push) Has been skipped
CI / build (push) Has been skipped
CI / dry-activate (push) Has been skipped
CI / deploy (push) Successful in 3m6s
26db6eaa16
exact revert of the eval-only stub + LDAP wheel/systemd-journal posixGroups.
sudo works via NSS group merge in principle (files+sss merges for groups,
unlike the first-match-wins passwd lookup), but something in the actual
runtime path - sssd cache staleness, ldapadd being create-only so the GID
fix never took effect on the live groups, or the sssd restart itself -
broke sudo on verdandi hard enough to need two generation-rollback reboots
to recover. the underlying idea (LDAP+Kerberos as sole auth, no local
account) is sound and sssd itself works fine for identity/read-only
resolution, but the interaction with sudo/group resolution needs a lot more
testing in a low-stakes environment before touching a host's actual access
control again. local lamentUser (password, ssh keys, extraGroups) is back.
lament merged commit 26db6eaa16 into main 2026-07-31 00:19:30 -05:00
lament deleted branch sssd 2026-07-31 00:19:30 -05:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
lament/pantheon!119
No description provided.