Troubleshooting
Fix startup, connection, submission, IPC and automatic update problems.
Startup errors
Read journalctl -u swqos or the terminal's error: lines. The --canary check helps separate configuration and network problems from listener failures.
| Error | What to do |
|---|---|
unknown argument or --config requires a path | Use --config /etc/swqos/swqos.toml, optionally followed by --canary. |
read config | Check the path and make the file readable by the swqos user. |
parse config or the customer binary does not accept the config option | Use the supported configuration. Keep upstreams above table headers. |
upstreams entry ... is not host:port with a host under sol.zerocopy.sh | Use a regional host name and port. |
invalid log filter | Fix log.level or the overriding RUST_LOG environment variable. |
Address already in use | Stop the other process or give each agent different listener ports. |
bind IPC ... No such file or directory | Create the socket directory or use the service example. |
bind IPC ... Permission denied | Make the socket directory writable by swqos. |
another instance holds the IPC socket lock or another instance is listening on the IPC socket | Stop the other agent or choose a different IPC path. |
IPC path exists and is not a socket owned by this user | Inspect the existing path. Remove it only if it belongs to a stopped instance, or choose a new path. |
Keep IPC paths shorter than 108 bytes. For no valid release slot or this build has no API key compiled in, check the executable's checksum and contact support for the correct licensed file.
The canary fails
For no region answered in time, check these from the agent's host:
- Region host names resolve to IPv4 addresses.
- Outbound UDP 443 reaches the region hosts.
- The system clock is correct for TLS checks.
- Any pinned
upstreamsentries name available regions.
If the connection is refused despite correct network access, contact support to check the license and API key.
Sends fail
| Rejection | What to do |
|---|---|
not_ready, waiting_for: "upstream" | Check /health, the service log and network access. |
not_ready, waiting_for: "tip_vaults" | Fetch tip accounts again shortly. |
missing_tip | Include at least the returned minimum using the current tip accounts, or your configured declared-tip arrangement. |
payment_required | Resolve the outstanding payment with support. |
forward_failed | Check the signature before retrying; see Confirmation and retries. |
unsupported | Query the tip floor through a hosted endpoint. |
See Errors for other refusals. Use requests with replies to diagnose problems; a request without a reply cannot report its rejection.
Local clients cannot connect
| Symptom | What to do |
|---|---|
| HTTP connection refused | Check that the service has finished starting and the address matches listen.http. |
| QUIC handshake fails | Use ALPN zerocopy-swqos/1 and the local certificate settings. |
IPC ENOENT | Check listen.ipc and that the service is running. |
IPC EACCES | Add the client account to the swqos group, then restart the client process. See IPC access. |
IPC ECONNREFUSED | Check that an agent is listening at the path, rather than a socket left by a stopped instance. |
IPC sessions end or replies are missing
After a stop or update, reconnect and repeat the handshake. A session that closes immediately after a submission may indicate an invalid record or ring; compare your client with the IPC wire reference.
For missing completions, check that each submission requests a reply with flag 0x01, uses a distinct request ID and that your client keeps draining completions. Set a timeout for every request. A timeout or closed session leaves the send outcome unknown; check the signature before resubmitting.
One CPU core is fully busy
With ipc_busy_poll = true, one core stays busy while any IPC client is connected, even an idle one. Set ipc_busy_poll = false, disconnect idle clients or restrict the agent's CPUs. See Busy polling.
Updates do not install
Check warnings with release= and error= fields:
installorPermission denied: make the executable's directory writable byswqos, including through service sandboxing.staged release failed its canary: collect the precedingerror:lines and contact support.restart <path>: check that the executable can run. The service example restarts it after a process failure.
See Updates for the host requirements.
The agent uses an unexpected region
Check the pinned region list, DNS and outbound UDP access. Small latency differences do not always trigger a switch. If you run many agents or direct QUIC clients with one API key, check the shared connection limit.
Contact support
Include your team name, the --canary output, relevant error: and log lines with their site= codes, and the agent configuration. Do not send the licensed executable or an API key in a support message.