engineering6 min read

What Sentry Found: Observability, Stability, and the Shifting Cost of Watching Your Own Code

Production errors exposed a schema mismatch, failed transaction recovery, and unhelpful logs. How a small team used Sentry to fix backend failures, prepare browser monitoring, and control alert noise.

K
Ken Jo

Observability used to be the kind of thing engineering teams either over-invested in (an entire SRE org wiring up dashboards nobody read) or skipped entirely (you find out the service is down because a customer emails you). In 2026, neither extreme is acceptable. You need to know what is breaking — but you also need to know without paying enterprise sticker prices for an indie product.

This walkthrough shows how our Sentry integration exposed production errors, what we changed in response, and how we configured monitoring to control noise and usage. It also looks at how observability tools are evolving for small engineering teams.

Why alerts were missing browser failures

Our backend has been wired to Sentry since March. The DSN sits in Google Secret Manager, the secret is mounted into Cloud Run, and sentry_sdk.init runs on every startup. Email alerts were arriving. From the outside, it looked fine.

From the inside, it was not. The frontend was not wired to Sentry at all. Browser exceptions, hydration errors, and chunked-load failures were invisible. And the backend was sending Python errors into a project labelled javascript-nextjs, because that is the only project in the organisation. Stack traces were piling up in the wrong place, grouping was useless, and the dashboard was a stream of mixed-language events.

Fixing that gap meant separating browser failures from backend errors before tackling individual bugs.

What the data showed

We pulled the last ninety days of error events. A few patterns stood out:

A historical incident that fixed itself

The most-frequent error in the window was a Postgres StringDataRightTruncation — sixteen events in twenty-two seconds — coming from the social signup endpoint. The cause was a classic schema drift: the model definition allowed an 800-character bio, but the production database column was still VARCHAR(500) from an earlier migration. A user submitted a bio just over 500 characters, our Pydantic layer accepted it, and the INSERT exploded.

The migration to widen the column to 800 had already been written and deployed the next morning. But the root problem — that model and DB schema can drift without anyone noticing until a customer trips it — was still wide open.

We did two things. First, we tightened the request schema for social signup: explicit max_length on email, display name, and token so anything pathological returns 422 cleanly instead of reaching the database. Second, we extended our DB integrity script to compare information_schema.columns.character_maximum_length against the lengths the application expects. Now if a future migration shrinks a column under what the model can produce, the next deploy fails the integrity check.

A session that would not roll back

Further down the list, a sequence: one OperationalError: server closed the connection unexpectedly on POST /v1/upload, followed by a stream of PendingRollbackError from the same code path. Postgres had dropped the connection mid-query, the SQLAlchemy session was now in an invalid state, and the upload service tried to do partial-result persistence on top of it. Every subsequent operation in that session cascaded the failure.

The fix is small but easy to get wrong. When the upload pipeline catches an exception, the very first thing it now does is db.rollback() — wrapped in its own try/except so a failing rollback cannot mask the original error. After that, the partial-save logic runs on a clean session.

Logs that erased themselves

The WebSocket finish handler was logging err=%s against a caught exception. For most exceptions that prints something useful. For a handful of exception types whose __str__ returns an empty string, it printed err= with nothing after. We were getting Sentry events that told us a save had failed without saying why.

Switching to logger.exception(...) and err=%r cost nothing and made every future event self-describing.

Noise we did not want to pay for

The remaining items in the queue were not bugs. Uvloop occasionally raises RuntimeError: unable to perform operation on <TCPTransport closed=True> when a client disconnects mid-SSL. That is correct behaviour, not something we can fix, and on a free-tier quota of 5,000 errors per month it is wasted budget. We added a before_send filter that drops events matching those specific exception/value patterns. Same idea on the frontend for ResizeObserver loop limit exceeded and similar browser noise.

Tuning for the free tier

Sentry's current Developer plan lists 5,000 errors, 5 million tracing spans and 50 replays per month for free, checked October 13, 2026. Tracing is measured in spans, not the legacy performance-unit allowance. Check the current plan limits before choosing sampling rates: an error allowance is not a budget for every signal the SDK can send.

Our calibrations:

  • tracesSampleRate: 0 on both backend and frontend. Distributed tracing is genuinely useful, but its span volume needs a separate measured budget. We can turn it on with a tight sample rate when we need to investigate something specific.
  • profilesSampleRate: 0 on the backend. Same reasoning.
  • Session Replay disabled on the frontend. We left the integration code in place but set both sample rates to zero. If a critical bug needs replay context to debug, we can flip replaysOnErrorSampleRate to 0.05 for a day and turn it back off.
  • before_send filters on both runtimes to drop the events we already know are not actionable.

The backend changes reduced actionable-error noise, while the browser SDK configuration was prepared for activation. Those are different milestones: code and sampling settings alone do not prove that browser events are reaching the intended project.

What's changing in the observability market

We tuned for the free tier because Sentry's free tier is unusually generous for a hosted product, but it is worth understanding the broader picture, because the economics here are moving.

A few things have shifted in the last twelve to eighteen months:

  • Open standards reduce migration work, not all migration work. OTLP-compatible instrumentation can reduce coupling to a monitoring backend. Vendor-specific queries, dashboards, alert rules, and SDK features still need planning and testing when you move.
  • Self-hosting changes the cost calculation. Options such as GlitchTip and OTLP-native stacks offer another route to monitoring. Compare storage, backups, upgrades, and engineering time with the hosted plan, not just a VM bill with a subscription price.
  • AI-assisted debugging extends the workflow. Sentry's Seer proposes root causes and patches, while an authorized MCP integration can bring error context into a coding assistant. A proposed fix still needs reproduction and regression tests before release.
  • Plan limits deserve the same attention as features. Error volume, retention, sampling, and team access can affect the bill. Start with the required signals and a measured event budget, then compare plans against that workload.

At this release snapshot, Sentry covered our immediate error-monitoring needs. Using open interfaces where available gives the team more options later, but moving vendors would still require checking project routing, alerts, retained data, and any provider-specific features.

Deployment checks complete the monitoring setup

At this post's May 17, 2026 snapshot, two deployment tasks remained:

  1. Create a dedicated Python Sentry project so backend errors stop being filed under javascript-nextjs. The DSN gets swapped in the same Secret Manager entry the backend already reads from.
  2. Wire the frontend DSN into Vercel's project settings so client-side errors start flowing in. The code is already there and conditionally-initialised — the only thing missing is the environment variable.

For any monitoring rollout, send a controlled test event from each runtime and confirm its project, environment, grouping, and context before calling the integration complete. Separate projects and an environment variable are setup steps, not evidence that the events arrived.

Why this matters

When you cannot see what is breaking, you over-invest in defensive coding, or you under-invest and ship bugs. Both are expensive. Cheap, reliable observability lets a small team behave like a larger one: catch regressions before customers notice, fix them without guesswork, and spend the engineering hours on building new things.

The useful lesson is to verify both halves: the application's fixes and the monitoring path that should reveal the next regression.


← Back to Blog