Root cause analysis in support: symptom ≠ cause

"The website has been extremely slow since this morning." That is how a typical ticket starts. The first look goes to the web server, the PHP process, the cache. But the real cause is often somewhere else entirely – for example on another server the customer never sees.

Symptom and cause are two different places

A symptom is the point where a person notices a problem. The cause is the point where it begins. Several systems often sit in between. Three typical patterns from everyday support:

The website is slow – cause: a full disk on the database server. The database can no longer write properly, queries hang, and the web server waits.
Emails don't arrive – cause: an expired certificate on the relay. The sender's mail server works fine; only the TLS connection to the next hop fails.
The login fails – cause: a clock running three minutes behind. Time-based mechanisms such as TOTP codes or SAML assertions with a validity window tolerate very little drift.

In all three cases, restarting the affected service would be useless or help only briefly. The ticket might be closed, but the problem would not be solved.

Workaround or cause?

Workarounds are not bad. If a customer cannot work, restoring service comes first. A service restart, a manually cleaned directory, a temporarily changed setting – all legitimate.

What matters is calling it what it is: "This is a workaround." Then comes the second question: why did it happen, and what stops it from happening again? A workaround closes the ticket. Only root cause analysis prevents the next one.

A simple approach

Finding the cause doesn't require a heavy framework. These steps work well:

Pin down the time. Since when exactly? A timestamp turns a vague description into a trail in the logs.
Ask what changed. Updates, deployments, new DNS records, expired certificates, maintenance at the provider. Systems rarely break for no reason.
Walk the chain. From the user through DNS, network, web server, application and database down to storage. At each link, check: does it work in isolation?
Ask "why?" several times. The website is slow – why? The database doesn't respond – why? The disk is full – why? A log isn't being rotated. That's where a lasting fix belongs.
Confirm the cause. A hypothesis only becomes the cause when fixing it actually removes the symptom.

The first minutes on a Linux server

Many causes can be found with a handful of standard commands before going deeper:

df -h                          # disk space
df -i                          # inodes
timedatectl                    # time and NTP synchronisation
systemctl --failed             # failed services
journalctl -p err --since "1 hour ago"
uptime                         # load and uptime

For certificates on a remote service, check the expiry date:

openssl s_client -connect mail.example.com:465 -servername mail.example.com </dev/null 2>/dev/null \
  | openssl x509 -noout -enddate

This doesn't replace an analysis, but within two minutes it rules out the most common suspects: full disks, wrong time, stopped services, expired certificates.

Document it so it doesn't come back

A good root cause analysis ends with a few clear sentences: what was the symptom, what was the cause, what was done in the short term, and what changes permanently? For example disk monitoring, a reminder before the certificate expires, properly running time synchronisation.

That note helps the colleague who sees the same case in three months – and often yourself as well.

Checklist

Exact time of the first occurrence recorded?
Changes in the period before checked?
Whole chain checked, not just the place of the symptom?
Disk space, time, services and certificates ruled out?
Workaround named as such?
Cause confirmed by the fix?
Permanent measure and short documentation written down?

The extra half hour spent on the cause feels like a luxury in the moment. Measured over weeks, it is the cheapest time you can invest in support.