It is simple, intuitive, and easy to compare. But that simplicity can become a problem when the data is highly imbalanced.

Failure detection is a good example.

Imagine a dataset with 10,000 observations:

• 9,800 are normal

• 200 are failures

Now consider a model that predicts every observation as normal.

It correctly classifies 9,800 observations and achieves 98% accuracy.

But it detects none of the failures.

Why Class Imbalance Matters

In many real-world failure detection problems, failures are much less common than normal behavior.

A spacecraft doesn't fail every time telemetry is collected. A server doesn't crash on every request. A manufacturing machine doesn't produce defective components most of the time.

Normal behavior dominates the dataset.

This creates a class imbalance between:

Normal observations → majority class

Failure/anomaly observations → minority class

When one class vastly outnumbers the other, accuracy can be dominated by the majority class.

If 99% of the observations are normal, a model that predicts "normal" almost everywhere can achieve close to 99% accuracy without actually being good at detecting failures.

So the important question isn't only how many predictions the model gets right.

It is whether the model is actually detecting the minority class.

What Accuracy Hides

A model can make two important types of mistakes:

• False positive: normal behavior is classified as a failure.

• False negative: an actual failure is classified as normal.

These errors don't necessarily have the same consequences.

A false positive may result in an unnecessary alert or inspection. A false negative, however, means that an actual failure has gone undetected.

That difference matters when evaluating a failure detection system.

A model with slightly lower overall accuracy may therefore be more useful if it detects substantially more of the failures we care about.

Looking Beyond Accuracy

This is where precision and recall become important.

Recall tells us how many of the actual failures the model detects.

Precision tells us how often the failures identified by the model are actually failures.

Looking at both gives us a much better understanding of how the model behaves on the target class.

A model with high recall but very low precision may generate too many false alarms.

A model with high precision but low recall may identify failures accurately when it raises an alert, but miss too many failures altogether.

The right balance depends on the system and the consequences of these errors.

A Practical Example

I encountered this problem while working on supervised anomaly detection for NASA SMAP spacecraft telemetry during my research internship at IIT Patna.

The dataset had severe class imbalance, so accuracy wasn't enough when comparing the models. I evaluated approaches including One-Class SVM, Random Forest, and SVC, and optimized the final pipeline using RandomizedSearchCV and SMOTE.

The final model achieved 88% accuracy, but that number alone didn't tell me how well it was detecting anomalies.

For the anomaly class, precision and recall were also important parts of the evaluation.

This was a useful reminder that the best-looking overall metric isn't necessarily the best indicator of whether a model is solving the problem it was built to solve.

Accuracy Isn't Useless

None of this means accuracy is a bad metric.

When the classes are reasonably balanced and different types of errors have similar consequences, accuracy can be a useful measure.

The problem is relying on it as the only measure of performance when the data is heavily imbalanced and the minority class is the primary concern.

Model evaluation should reflect the actual objective of the system.

For failure detection, that means asking not only whether the model is usually correct, but whether it can reliably identify the failures that matter.

The Bigger Lesson

A model can look successful according to a metric and still perform poorly at the task we built it for.

The metric isn't necessarily wrong. Sometimes, we're simply asking it to answer the wrong question.

So don't stop at the accuracy score. Ask:

Is the model actually good at detecting the failures?

That is what the system was built to do.

Let’s say you have a very long URL and want a shorter version of it so that it is easier to share or remember. How can you do that?

That’s where a URL shortener comes into the picture.

It takes a long URL and generates a short code by internally mapping that short code to the original URL. Whenever someone accesses the short code, the system looks up the original URL and redirects the user to it within a very short amount of time.

That’s it.

At least, that’s how I initially thought about it.

I wanted to build a simple URL shortener, so after a little planning, I started building Routr.

Starting with the basic version

I first created the URL model with the attributes I thought each URL needed: a unique ID, the original long URL, the short code mapped to it, and other relevant information.

Then I built the two basic operations:

• A POST endpoint to create a short URL.

• A GET endpoint to redirect a short code to its original URL.

For persistence, I used PostgreSQL to store and retrieve the URL mappings.

At this point, I had the basic URL shortener working.

But while building the initial prototype, I ran into my first interesting problem.

How do I generate unique short codes?

Every new URL needs its own short code. So how should I generate one?

I started experimenting with different approaches. One of the approaches I remember trying was hashing the long URL and using some part of the resulting hash as the short code.

The idea works, but there is a problem: collisions are possible. Two different URLs can theoretically produce the same hash or the same shortened portion of a hash.

I didn’t want to introduce collision detection and resolution just to generate a short code.

Eventually, I came across Base62 encoding.

Base62 uses lowercase letters, uppercase letters, and numbers to represent an integer in a compact form. This gave me a much simpler approach.

Instead of trying to generate a unique value from the URL itself, I could use something that was already guaranteed to be unique: the database-generated ID.

So the flow became:

Database ID → Base62 encoding → Short code

Since every row gets a unique ID from the database and Base62 is simply encoding that integer, different IDs produce different short codes.

That solved the short-code generation problem without having to deal with hash collisions.

Then I added analytics

After getting the basic functionality working, I thought: what if users could also see how many times a short URL had been accessed?

So I added click tracking.

I updated the URL model to store the click count and modified the redirect logic to update the count whenever someone accessed a short URL.

The feature worked.

But while testing it, I was also logging the requests being made, and I noticed something interesting.

Every redirect was causing a database read to retrieve the original URL and a database write to update the click count.

For one request, that is obviously not a problem.

But what happens if a large number of requests start hitting the same URL?

I started thinking: can I reduce the number of reads and writes going to my main database?

That’s when Redis came into the picture.

Introducing Redis

I decided to use Redis as a cache in front of PostgreSQL.

The idea was simple: if a short URL has already been requested recently, why should I query PostgreSQL again for the same information?

So I started storing the mapping:

short code → long URL

in Redis.

Now, whenever a redirect request comes in, the application first checks Redis.

If the value is found, we have a cache hit:

Redis → long URL → redirect

No PostgreSQL lookup is required.

If the value isn’t found, we have a cache miss:

Redis miss → PostgreSQL → get long URL → store in Redis → redirect

The next request for the same short code can then be served from Redis.

This is essentially the cache-aside pattern.

But what happens when the cache becomes full?

Once Redis was working, another question came up.

If I keep putting URLs into the cache, won’t it eventually keep growing?

I obviously don’t want every URL that has ever been created to stay in Redis indefinitely.

So I introduced a TTL of one hour for cached URL mappings.

The idea was that cached entries should not remain there forever. After the TTL expires, the entry is removed, and if somebody requests that URL again later, it can be fetched from PostgreSQL and cached again.

This gave me a simple way to prevent cached entries from accumulating indefinitely.

But analytics had another problem

I had solved the repeated database reads, but there was still the analytics problem.

Every redirect was still incrementing the click count.

I started thinking: do I really need to write to PostgreSQL after every single click?

Probably not.

Instead, I moved the live click counter into Redis as well.

Now every redirect increments the counter in Redis rather than immediately updating PostgreSQL.

Then I added a scheduled synchronization process that periodically takes the accumulated Redis count and adds it to the persisted count in PostgreSQL.

For example, if PostgreSQL has:

1,000 clicks

and Redis currently has:

250 clicks

the synchronization updates PostgreSQL to:

1,250 clicks

and resets the Redis counter back to zero.

There was one more thing to consider.

What if someone asks for the analytics before the next synchronization happens?

In that case, PostgreSQL might say 1,000 while Redis has another 250 live clicks.

So when returning analytics, I combine the persisted PostgreSQL count with the current Redis count to give the latest available number.

The final architecture

So what started as a simple URL shortener gradually evolved into something like:

PostgreSQL handles persistent data.

Redis handles frequently accessed URL mappings and live click counters.

Cache-aside keeps repeated URL lookups away from PostgreSQL.

TTL prevents cached URLs from remaining indefinitely.

Scheduled synchronization moves accumulated analytics from Redis back into PostgreSQL.

The interesting part for me wasn’t implementing Redis itself.

It was seeing how one decision created another problem, which then led to another architectural decision.

I started with:

URL shortener → PostgreSQL

Then:

Need unique short codes → Base62 + database ID

Then:

Analytics → database writes on every click

Then:

Repeated database access → Redis cache

Then:

Cache growth → TTL

Then:

Too many analytics writes → Redis counters + periodic synchronization

And that’s probably the biggest thing I learned from building Routr.

I don’t think a system needs to start with a complicated architecture just because it might eventually have a lot of traffic. Start with something simple, understand how it behaves, identify the actual problems, and then introduce complexity when there is a reason for it.

I haven’t benchmarked Routr yet, so I can’t claim a specific percentage reduction in latency or database load. That’s something I want to do next.

For now, the project gave me something more valuable: a better understanding of how backend architecture evolves as you start asking, “What happens if this gets bigger?”

Try Routr →