Follow-up: root cause narrowed down — vhost_virtual panics on SSL renewal timer after creating a site under a new sub-account
Environment
- Ubuntu 22.04 VPS, aaPanel with Account Management (sub-panel) enabled
- Backend service:
vhost_virtual (/www/server/vhost_virtual/vhost_virtual), systemd unit vhost_virtual.service
Symptom as first observed
The Account Management page returned:
Request failed: HTTPConnectionPool(host='127.0.0.1', port=xxxx): Max retries exceeded
with url: /log/get_logs (Connection refused)
The firewall listed TCP xxx as "Not Listening". This turned out to be a red herring — nothing was bound to the port because the backend service was dead, and loopback traffic isn't firewalled anyway.
What's actually happening
vhost_virtual.service starts cleanly, binds :xxxx, serves requests, then exits with status=2/INVALIDARGUMENT roughly 15–30 seconds later. Every time, reproducibly. There is no Restart= directive in the unit file, so the service stays down until manually started — which is why the panel page appears permanently broken.
The application log shows a Go panic in the Let's Encrypt auto-renewal timer:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18]
goroutine 129 [running]:
VirtualHost/internal/service/public.GetAccountInfo(...)
internal/service/public/common.go:3098
VirtualHost/internal/model/site.ApplyLetsencryptAt(...)
internal/model/site/ssl.go:251
VirtualHost/internal/model/ssl.processSSLRequest(...)
internal/model/ssl/ssl.go:110
VirtualHost/internal/model/ssl.ApplyLetsEncryptAndDeploy()
internal/model/ssl/ssl.go:192
VirtualHost/internal/service/ssl.NewTimerEvery.func1()
internal/service/ssl/ssl.go:19
GetAccountInfo() at common.go:3098 returns nil and the result is dereferenced without a guard, taking down the whole process.
Trigger
Failures start on Aug 31, immediately after I created a new sub-account (filegatenz) and added a PHP site (storage.gatenzteam.com) under it via the Account Management panel. Prior to that the service was stable.
Two secondary issues visible in the same startup log
- A missing SQLite path — the email module cannot initialise:
Failed to create table [ email ]: CREATE TABLE IF NOT EXISTS domain_user (...)
: unable to open database file: no such file or directory
There is no email.sqlite in /www/server/vhost_virtual/data/db/ alongside the other databases. This may well be why GetAccountInfo() returns nil.
- A schema definition that looks wrong: in
website.sqlite, table sites declares
cert_id INTEGER NOT NULL DEFAULT ''
An INTEGER column with an empty-string default. Existing sites carry integer values (1, 2), but the newly created site picked up the empty-string default. Setting it to 0 did not stop the panic, so this isn't the direct cause — but the type mismatch still looks unintended.
Data integrity checks that came back clean
- Every
sites.account_id resolves to an existing row in account.sqlite — no orphaned records
business_cert is empty- The two rows in
letsencrypts are CloudFlare Origin Certificates, not ACME-issued certs. Worth checking whether the renewal timer handles non-ACME entries in that table correctly.
What I'd suggest the team look at
- Add a nil check at
common.go:3098 — regardless of the underlying cause, a background timer should not be able to segfault the entire service.
- Review site creation under a sub-account in Account Management — something about that path leaves state the SSL renewal timer can't handle.
- Check why
email.sqlite / its parent path is missing on a working install.
- Add
Restart=on-failure to the unit file — though only after the panic is fixed, otherwise it would just crash-loop.
- Review the
cert_id INTEGER NOT NULL DEFAULT '' declaration.
Happy to provide further logs or run diagnostics if that helps.
