The authelia configuration contains some sensitive values, but authelia cannot read them from env variables like traefic can. Instead, a special service - authelia-config runs before authelia start, and preprocesses the configuration file.

Local file ./authelia/configuration.yaml is mapped to /data/input in authelia-config
Volume authelia-config is mapped to /data/output in authelia-config
authelia-config runs gomplate on /data/input and saves to data/output/configuration.yaml
Volume authelia-config is mapped to /etc/authelia in authelia, where it reads its configuration

Lessons learned

Authelia will ONLY work with https. Both the authelia url itself and the one being authenticated must be https.
The authorization link should NOT end with /#/ or /%2F/ or anything, just /. Otherwise it will not redirect you back after authorizing.