Skip to content

Latest v0.51.0 breaks my setup #1998

Closed

Description

@driftywinds

Here are the logs from my container -

Charon v0.51.0 (commit d034513951edebbdb2840e1e5bbd51b3b8491799, built 2026-11-16T21:15:57Z)
Docker Compose, non-root container, CrowdSec disabled in settings

--- startup ---
{"level":"info","msg":"starting Charon backend on version 0.51.0 (commit: d034513951edebbdb2840e1e5bbd51b3b8491799, built: 2026-11-16T21:15:57.152Z)","time":"2026-11-17T00:41:55Z"}
{"level":"warning","msg":"CHARON_EMERGENCY_TOKEN not set - emergency bypass disabled","time":"2026-11-17T00:41:55Z"}
{"level":"info","msg":"auth throttle: login 10/600s, session 60/60s per client; trusted proxies: 0","time":"2026-11-17T00:41:55Z"}

--- every proxy host is skipped (repeated ~70 times, one per host UUID) ---
{"host":"689cdc87-xxxx-xxxx-xxxx-xxxxxxxxxxxx","level":"warning","msg":"Skipping proxy host: its forward target is not permitted by this proxy","time":"2026-11-17T00:41:57Z"}
{"host":"8810016e-xxxx-xxxx-xxxx-xxxxxxxxxxxx","level":"warning","msg":"Skipping proxy host: its forward target is not permitted by this proxy","time":"2026-11-17T00:41:57Z"}
{"host":"29806fcb-xxxx-xxxx-xxxx-xxxxxxxxxxxx","level":"warning","msg":"Skipping proxy host: its forward target is not permitted by this proxy","time":"2026-11-17T00:41:57Z"}
... (same message for all remaining hosts)

--- resulting Caddy config is tiny ---
{"level":"info","ts":1794876117.2874544,"logger":"admin.api","msg":"received request","method":"POST","host":"localhost","uri":"/load","headers":{"Content-Length":["11991"],"Content-Type":["application/json"]}}
{"level":"info","ts":1794876117.2998126,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["svc1.example.com","svc2.example.com","svc3.example.com","svc4.example.net","site.example.org","apprise.example.net","svc5.example.com","svc6.example.net","svc7.example.net","www.site.example.org"]}
{"level":"info","msg":"Successfully applied initial Caddy config","time":"2026-11-17T00:41:57Z"}

--- sample 502 from the one remaining host (upstream is 127.0.0.1) ---
{"level":"error","ts":1794876143.2258716,"logger":"http.log.error.access_log","msg":"dial tcp 127.0.0.1:3038: connect: connection refused","request":{"remote_ip":"203.0.113.10","client_ip":"203.0.113.10","proto":"HTTP/2.0","method":"GET","host":"apprise.example.net","uri":"/notify/[REDACTED]"},"duration":0.000485083,"status":502,"err_id":"mzitq3rh2"}

Going back to v0.50.0 instantly fixes it for me, so it is def a latest version problem

Activity

  1. self-assigned this
    on Nov 17, 2026
  2. moved this from Backlog to In Review in Charonon Nov 17, 2026
  3. Wikid82 commented on Nov 17, 2026

    @Wikid82
    Owner

    Thanks for the report, and for including the logs. They point at the cause.

    v0.51.0 added a check on forward targets when the Caddy config is built. A host whose target fails the check is left out of the config, with the warning you're seeing. The check doesn't stop at the host you were working on. One refused target leaves out just that host, and the rest still load. Seeing about 70 hosts skipped at once means something they share is being refused. I can't tell what from the logs, because they only show host UUIDs.

    A forward target is refused in these cases:

    • It's a literal IP address that is link-local (169.254.x.x, fe80::), unspecified (0.0.0.0, ::), multicast, a cloud metadata address, or an IPv6 transition range (6to4, Teredo, NAT64).
    • It uses the agent-name form (orthrus:...).
    • It points at this machine, by name or address, on a port Charon keeps for itself. That means the Caddy admin port, the emergency port, the CrowdSec local API port (8080 by default), or an agent proxy port. A name that can't be resolved is refused on those ports too.
    • Its advanced configuration can't be parsed, or uses dynamic upstreams or a custom resolver.

    To narrow it down, could you tell me:

    1. What the forward host and port look like for a few of the skipped hosts? Please mask the values if you like, but keep the shape: IP or hostname, which ports, and whether they're Docker container names, LAN IPs, host.docker.internal, or the Docker gateway IP.
    2. Whether any skipped hosts forward to the Docker host itself, for example 172.17.0.1 or host.docker.internal, on port 8080 or another common port?
    3. Whether the skipped hosts have anything in Advanced Config, such as custom upstreams or resolvers?
    4. Whether the hosts are plain proxy hosts, or Orthrus agent hosts, or a mix?
    5. Your Docker network mode (bridge, host, macvlan) and your compose file with secrets removed.

    If you can, run this against the data directory with the container stopped, or through docker exec with a SQLite client. It lists the stored targets without the domains:

    sqlite3 /app/data/charon.db "select forward_host, forward_port, enabled from proxy_hosts limit 20;"
    

    Rolling back to v0.50.0 was the right workaround for now. I'll keep this open and look into it once I have those details.

  4. driftywinds commented on Nov 17, 2026

    @driftywinds
    Author
    1. Forward host/port shape
      almost all of my proxy hosts use 0.0.0.0 as the forward host with a per-service port, for example:
    • 0.0.0.0:3038
    • 0.0.0.0:[port]
    • 0.0.0.0:[port]

    no Docker container names, no host.docker.internal, and no 172.17.0.1. The services run on the same machine as Charon and publish their ports on the host.

    1. Forwarding to the Docker host itself?
      no. everything is 0.0.0.0:<port>. There might be some using 127.0.0.1

    2. Advanced Config
      nope

    3. Host types
      plain proxy hosts only, no Orthrus agents.

    4. Docker network mode
      host

  5. Wikid82 commented on Nov 17, 2026

    @Wikid82
    Owner

    That explains it, thank you.

    0.0.0.0 is the "unspecified" address, and the v0.51.0 forward target check treats it as a blocked address. The check allows loopback (127.0.0.1) and private LAN ranges, but 0.0.0.0 isn't in either group. So every host using it is left out. Any host on 127.0.0.1 stays, which matches your single remaining host. On Linux, connecting to 0.0.0.0 reaches this machine, so it behaved like loopback before, and this is a regression rather than intended behaviour.

    Workaround until a fix ships: change the forward host on those hosts from 0.0.0.0 to 127.0.0.1 (same ports). Or stay on v0.50.0.

    I'm working on a fix that treats 0.0.0.0 the same as loopback for forward targets. I'll link the PR here.

  6. moved this from In Review to Done in Charonon Nov 17, 2026
  7. moved this from Done to In Progress in Charonon Nov 17, 2026
  8. driftywinds commented on Nov 17, 2026

    @driftywinds
    Author

    Can I update freely, will it auto handle 0.0.0.0

  9. Wikid82 commented on Nov 17, 2026

    @Wikid82
    Owner

    Yes, you can update freely. v0.51.1 includes the fix (#2000), so 0.0.0.0 forward targets are handled automatically. Your stored hosts don't need editing, and you can keep 0.0.0.0 as the forward host. After the upgrade, all your hosts should load again on startup and the "Skipping proxy host" warnings should be gone.

    Two things to know:

    • Charon treats 0.0.0.0 like 127.0.0.1, so it is still refused on ports Charon keeps for itself (the Caddy admin port, the emergency server port, the bundled CrowdSec port and agent proxy ports). A service on one of those ports would still be skipped. If you see any "Skipping" warnings after upgrading, tell me the port.
    • Uptime monitors and Test Connection still refuse 0.0.0.0. If you use them on these hosts, use 127.0.0.1 for the monitor target.

    Thanks for the clear logs and for answering so quickly. Let me know if anything is still skipped after the upgrade.

  10. moved this from In Progress to In Review in Charonon Nov 17, 2026
  11. moved this from In Review to Done in Charonon Nov 18, 2026
  12. closed this as completedby moving to Done in Charonon Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

backendServer-side codecaddyCaddy-specificcrowdsecCrowdSec integrationsslSSL/TLS certificatestestingTest suite

Projects

  • Status
    Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions