A platform engineering team is doing a great job until customers start complaining about why their applications feel slow. Then things get real and someone must find out if the issue is in the application's own code or in the platform running it.
This is a common scenario that appears most in large, managed setups like serverless platforms. Customers provide their code or containers, and the platform handles execution, scaling, fault tolerance, and traffic. Instances scale up in seconds during a traffic spike and down to zero when demand stops. This is the epitome of reliability without operational work.
Global product launches, major sports events, seasonal peaks, and breaking news create sudden surges in requests. Traffic can grow tenfold in minutes. The platform must absorb that load, route traffic across regions, start new instances, and balance requests across a worldwide network.
All of this happens fast and continuously. Every step of a request’s path is timed, logged, and measured. The platform records when traffic enters, when it is routed, when an instance starts, and when it reaches the customer's code. At a large scale, this creates a huge volume of data, enough to reconstruct the system's behavior in fine detail.
Yet with all that data, one simple question remains hard to answer: who caused the problem?
Why Slowness Is Hard to Diagnose
When a customer reports a slow application, there are many possible reasons. The slowdown might come from a recent change in the code. In a big company, that change could have been made by another team, or caused by a software dependency that acts differently under load.
The traffic itself may have changed. A new mobile app release can spike usage patterns overnight. A new feature can create new request types. Even if the application does not change, downstream systems like databases or outside services can add delays.
At the same time, the platform itself is always changing. Infrastructure teams deploy updates, add optimizations, and adjust how work is scheduled. A "noisy neighbor" on shared hardware can use too much memory or CPU and affect other workloads. How workloads are packed onto a machine can push it closer to its limits than planned. None of these issues are visible to the customer, and none are easy to connect quickly.
The result is a common pattern. A single ticket reports slowness. Nothing is alarming at the platform level, so the instinct is to think the issue is isolated. Then more tickets arrive, calls get longer and dozens of engineers join to check logs and compare timelines. Sometimes the investigation ends with the uncomfortable fact that the platform itself caused the problem.
The next question is obvious. If the platform already collects so much data, why does it need customers to report these issues? Why can't it detect platform-caused latency before users notice?
The Limits of Simple Latency Service Level Objectives (SLOs)
A natural starting point is to treat latency as a service level objective. Measure request latency, set a speed limit, and declare anything slower a violation. If too many requests cross the line, the platform must have gotten worse.
In reality, this approach fails quickly. Latency in large systems varies widely. Customers tolerate a surprisingly wide range of response times depending on the task. A single global speed limit does not match how users actually experience performance.
Set the limit too strict and the platform triggers constant false alarms. Set it too loose and real problems disappear in the noise. Worse, raw latency mixes platform behavior with application behavior. A slow request may say nothing about the platform at all.
This makes simple latency-based SLOs a blunt tool. They produce numbers, but not understanding.
Isolating the Platform’s Responsibility
A more practical approach begins by narrowing the measurement to what the platform directly controls. One important segment is the time it takes for a request to travel from the edge of the infrastructure to the moment it is handed to customer code. This "request delivery latency" captures routing, scheduling, and infrastructure performance without mixing in application logic.
By isolating this phase, the platform can see changes in its own behavior. When request delivery latency changes, it shows that something is going on in the underlying system. That change may not yet be visible to customers, but it is an early warning that reliability might be dropping.
This distinction arguably helps you to create a better, customer-centered view of performance. Instead of reacting to tickets, the platform can start to detect problems as they form, measure their impact over time, and see if they come from internal changes or outside factors.
Measuring Slowness Without Letting Traffic Hide the Problem
Once request delivery latency is isolated, it is easier to see if the platform is behaving correctly. The range is still wide, but narrow enough to set a meaningful threshold. Requests slower than an expected number of milliseconds can be counted, and their ratio tracked over time.
At first, this looks good. If too many requests exceed the threshold, something is wrong. But in practice, combining all the data creates a new problem. Large customers create enough traffic to drown out everyone else. One unhappy, high-volume workload can dominate the signal even if the rest of the platform is healthy. The opposite can also happen meaning that a large, stable workload can hide problems affecting many smaller customers.
Breaking the data down by customer and by infrastructure cell improves visibility. Each cell behaves a bit differently, and grouping by these boundaries shows localized issues. When this view is plotted over time, a baseline appears. Some customers consistently sit above the threshold without complaining. They have adapted to that level of performance; it is their normal state.
The signal appears when the normal baseline starts to shift. A sudden increase in the number of affected customers within a cell shows a real regression. The scale of the increase shows the impact immediately. This approach answers an important question: how many customers are affected, and where?
Still, it only captures one slice of the request lifecycle. It does not describe the end-to-end experience, and it cannot be applied the same way to every processing phase. To go further, the definition of reliability itself needs refinement.
Redefining Reliability Around Customer Experience
Reliability rests on three foundations: availability, performance, and correctness. Each is deceptively hard to measure at scale.
Availability is often shown as a ratio of failed requests to total requests. In large platforms, this metric becomes unclear. Errors can come from user behavior, from retries, or from timeouts without clear error codes. Failures that look like client issues may be triggered by the system itself while raw error rates lack precision.
Performance metrics face similar challenges. Synthetic tests measure specific scenarios, but they cannot represent every real workload. Tail latency SLOs are useful, but they still rely on thresholds that usually don’t match customer perception.
Correctness relies heavily on testing. Extensive test suites prevent obviously bad changes from reaching production, but tests do not capture every real-world interaction. They reduce risk, not uncertainty.
What customers notice most strongly is change. They rarely complain when a system gets faster. They notice immediately when it gets slower. Their expectations are shaped by past performance. When latency moves away from what they are used to, reliability feels broken even if the numbers are still within stated limits.
That observation leads to a more practical framing. Reliability, as customers experience it, is the stability of performance over time.
Detecting Change Instead of Chasing Absolute Thresholds
Each workload has its own latency profile. What feels slow for one service may be fine for another. From the customer’s view, both are reliable as long as performance stays consistent. This makes stability measurable. If a workload’s latency pattern remains statistically similar over time, performance is predictable.
The platform treats each deployed version of a service as a group. For every group, it models the latency pattern and saves the parameters as a baseline. Incoming requests are then checked against that baseline using statistical testing. Each request gets a score for how likely it is to belong to the expected pattern.
Rather than tracking raw latency values, the system tracks how unusual each request is relative to that workload's normal behavior. This produces a timeline of outliers, not milliseconds.
Importantly, change is measured in standard deviations. This normalizes performance across different workloads. A slow service and a fast service are evaluated using the same unit of change. This makes shifts easy to spot, even when actual latencies are very different.
The Two-Sigma Signal
Statistical theory gives a useful reference. In a stable pattern, only a small percentage of measurements should fall more than two standard deviations from the average. When that percentage rises sharply, the idea of stability breaks down.
The platform tracks the share of requests that cross this two-sigma boundary. When outliers climb beyond expected levels, it signals a regression customers will likely notice. This method creates a strong signal without arbitrary speed limits.
From there, the data becomes operational. For each project, the platform calculates the percentage of outlier requests. These values are combined to show how many customers are experiencing unusual latency at the same time. A spike in affected projects points to a platform issue, not isolated application behavior.
This allows for early detection. By the time a significant share of projects shows high outliers, users are already feeling the impact even if they have not filed tickets. The system can surface problems before alarms fire and escalation begins.
From Detection to Impact Assessment
When an alert goes off, figuring out the damage is usually pretty simple. Engineers can quickly assess the scope of the problem, pinpointing the number of projects impacted and the extent of the disruption. A sudden increase in activity could suggest a problem in a specific area or cell while a broad increase suggests a general decline.
This clarity helps speed up the decision-making process. Teams can confidently declare incidents, bringing in the appropriate personnel and taking action, such as rerouting traffic from trouble spots. The emphasis goes from “Do you have a problem” toward actually acknowledging it and resolving it.
The background noise persists. Statistical models are approximations, and not every workload fits perfectly. The main point is the difference between the initial state and the change. A sudden spike in the signal often indicates genuine customer distress.
Strengths, Limits, and Practical Use
This approach works without training data or manual tuning. It uses the statistical properties of observed behavior and checks itself against past outages. In practice, it reliably detects historical incidents and scales across large systems.
It does have limits. Very low-traffic workloads do not produce enough data for a reliable model, and their latency expectations are loose anyway. The method also focuses on symptoms, not causes. It shows that customers are affected, but not which component is responsible.
That limitation is intentional. Symptom-based monitoring reflects what users experience. Root cause analysis comes next, aided by correlation. By using similar models on individual phases of request processing and on infrastructure parts, engineers can find where latency increased in line with customer impact.
This pattern extends beyond monitoring. The same signals support experimentation, letting teams compare performance stability across rollout groups and catch regressions during controlled updates.
A Practical Definition of Platform Reliability
What emerges is a measurable definition of reliability based on customer perception. Stable latency becomes the signal. Deviations become actionable. Impact becomes visible in terms that matter to both engineers and leaders.
The result is not perfect observability, but a very useful and reliable way to know when customers are being affected, how many of them feel it, and when to act before the problem grows.
That is the difference between reacting to complaints and operating a platform that understands its own reliability.
How Solwey Can Help
Solwey is a boutique agency established in 2016 focusing on customers' success through excellence in our work. Often, businesses require simple solutions, but those solutions are far from simple to build. They need years of expertise, an eye for architecture and strategy of execution, and an agile process-oriented approach to turn a very complex solution into a streamlined and easy-to-use product.
That's where Solwey comes in.
At Solwey, we don't just build software; we engineer digital experiences. Our seasoned team of experts blends innovation with a deep understanding of technology to create solutions that are as unique as your business. Whether you're looking for cutting-edge ecommerce development or strategic custom software consulting, our team can deliver a top-quality product that addresses your business challenges quickly and affordably.
If you're looking for an expert to help you integrate AI into your thriving business or funded startup get in touch with us today to learn more about how Solwey can help you unlock your full potential in the digital realm. Let's begin this journey together, towards success.
