Neil Kakkar

Agentic Debt

2026-02-24T08:00:00+00:00

In my second week at Tano, I tried to add a simple feature. I got stuck because Claude made changes in the wrong place. Turns out, there were three different places in the codebase using almost the same frontend code, displaying slightly different UIs. It made the change to one I wasn’t aware of, so when I tried running it locally, nothing changed where I expected.

I took a step back, refactored and reconciled the three different copies to create something I could understand. I deleted about 2,000 lines of code written just a few weeks prior. More critically, the agent one-shotted the feature after.

This feels like a new kind of problem. It’s not the classic technical debt - no human consciously chose to re-create three different UIs, all almost the same. It’s something more specific to how agents write code.

I’ve been calling it agentic debt.

The feedback loop

Unlike tech debt, agentic debt is self-reinforcing.

An agent writes code. It works. You ship it. The agent writes more code. Also works. You ship that too. But each time, it’s optimising for the task at hand: the PR, the feature, the immediate ask. It doesn’t have a model of “everything else” the way you do.¹

Over time, you get locally optimal choices that add up to global architectural drift. Patterns get duplicated. Abstractions get half-implemented. Three different files handle the same concept in three slightly different ways.

And then the next agent comes along and tries to work with this codebase. It gets confused. It makes sub-optimal choices. It picks a whacky connection to the “jungle” side of the codebase - code that’s not touched anymore. Because the first agent’s slop makes it harder for the second to reason clearly.

This is the agent slop feedback loop. Decay that feeds itself.

The context window trap

A potential counter-argument is that this is unnecessary when context windows are large enough to dump the entire codebase in. An agent can then figure everything out, since code is deterministic.

In a year or two? Maybe. Today, I don’t think so. Even with infinite context, I expect noise and inconsistency break reasoning. Three duplicated patterns with subtle differences don’t become clearer with more context. They become three times as confusing. The model has to figure out which one is “right”, or if they’re all slightly different for good reasons, or if the last agent just copy-pasted without checking.

Human-understandable is agent-actionable

At least for now, the counter-intuitive finding: The best way to improve agent performance is to make sure the code is simple enough for humans to model.

When I reconciled those three duplicated frontends into one clean pattern, I wasn’t doing it for the agent. I was doing it because I couldn’t build a clear mental model of the codebase with all that fragmentation. But once the code was clean, the agent could reason about it better too. Future changes became faster. Future refactors became easier.

This isn’t a coincidence. Agents struggle with the same things humans struggle with: inconsistency, implicit assumptions, and duplicated logic with subtle differences. A codebase that’s easy for a human to hold in their head is also easy for an agent to work with. The properties that make code maintainable haven’t changed just because the writer is an LLM.²

Tending the garden

This changes how I think about working fast. Are we trading shipping something new tomorrow for slowing down later?

Especially at startups, this can be the right trade-off. I think it was at Tano too. The point is to make it explicitly, not let it be the default. Sometimes, slower is smoother. And smoother is faster.

Some refactors become much easier with agents. Deduplication, for instance, is cheap labour now. But agility depends on whether you actually do the refactoring, or just keep sprinting.

A clean codebase means new people get onboarded quickly, which lets them steer agents better. We aren’t at a point yet where you can stop steering.

As the codebase grows, and you go from one to many engineers, stewardship becomes more important. Sure, you can hit approve, ship it, move on. But that slows down every future change. The slop shows up in how much attention the steerer pays.

You’re not necessarily writing the code anymore. But you are the maintainer. The gardener, if you will. Weeds will grow. Sometimes they’re needed to enrich the soil. Other times you need to trim them.

This is a different kind of engineering than what I was doing a few years ago. I used to write the code. Since I had to do the grunt work, I’d naturally make it easier for me to continue contributing. A lot of software best practices spun out of this.³

Now I’m the person making sure the code stays coherent. It’s less about “can I implement this feature?” and more about “does this feature fit into the system in a way that won’t confuse the next agent, or the next human?”⁴

Open questions

Model capabilities are improving pretty quickly. The scaffolding of today might be unnecessary tomorrow. There are a few things I’m not sure about.

Can a gardener agent work? A daily monitor that audits every PR and flags duplicated logic, unnecessary API endpoints, architectural drift, etc. etc. Codex reviews Claude pretty well, so I think this is plausible. I don’t know if it will be good enough though. It’s one experiment I want to try soon.
Code reviewer agents? With today’s models, I’m sure this doesn’t work well for stewardship. But in the future, I’m not sure. I think this arrives before agents that need no steering.
Does gardening scale? At Tano, we’re a small team. The gardening is manageable. But what happens when you have 50 agents writing code across a large codebase? Does the gardening overhead grow linearly, or does it compound? I suspect it compounds, which means the gardener role becomes even more important as we scale up agentic development.
When does refactoring pay for itself? Taking a step back to reconcile slop feels like lost momentum. But it was necessary preparation for the next burst of agent-driven velocity. I don’t have a good heuristic yet for when to pause and garden versus when to keep sprinting.

Until I figure it out, I garden between the sprints.

Even with enormous context windows. More on this later. ↩
I suspect this is because the models are trained on human-written code and human explanations. Their “reasoning” mirrors human reasoning patterns. What’s legible to us is legible to them. ↩
Which seems to suggest the best practices are changing. I think some core principles will remain, and some others will endure because humans steer the agent to such practices. What about a post-stewarding world, when agents are much smarter at coding than me? I don’t know ↩
There’s a parallel here to how choosing what to build has become more important than just building. Taste is more important, and more on this in a future post! ↩

What I learned about burnout and anxiety at 30

2026-02-16T00:00:00+00:00

If I had to choose one moment that signifies the deepest shift in the last few years, it would be mid-2024 when I had my first and only panic attack. It was my body and mind finally protesting and giving up, being like, “Neil, fuck this shit. You are not seeing the signs, so I have to take some drastic measures.”

It caught me completely by surprise. I did not know what was happening to me. It felt like a serious illness, and in a way I guess it was, just more mental than physiological.

Think of life like a four-legged stool. It’s super sturdy with four legs. It’s still pretty damn sturdy with three legs. Go down to two, and it’s hard to stabilise on most surfaces. At one, you’re definitely falling over, the only question is how slowly.¹ At zero, you don’t have a stool left, you’re a pancake.

I think these four legs are emotional connections, physical activity, hobbies, and work.²

And I, became a pancake.

The slow slide

I was deep into work. I became a manager, became responsible for high-profile client-facing features, and cared a lot about what we were building at PostHog. It was a wonderful time of growth, punching above my weight, learning a lot, but also doing too much of it. I was switched on all the time, and escaped into fantasy novels afterwards. It.. wasn’t healthy. And I didn’t realize it until it all came crashing down.

Thus began my long and slow road of dealing with anxiety. It was there long before the panic attack, but I did not understand what it was. It showed up as this compulsion to go back home, read some manga, and escape into fantasy novels. That felt.. safe.

I was also getting lonely. I had friends, but I rarely went out to meet them. Almost every time I did go to a social event, I adored it. Every time I had to decide to go to an event, I dreaded it. I thought I had too much work, and I wanted to relax after work by myself. I couldn’t be bothered to go out.

As the 2019 me thought about it, life is a complex adaptive system. Every part of life is an input to some other part of life. I theoretically understood it, but the visceral feeling of experiencing it is different.

The signs I missed

Observation is my superpower, but it’s a lot more retrospective than I would like. I’m not good at catching things in the moment, yet. It’s hard and it takes a lot of time.

I spent so long observing and tracking things that I stopped listening to my body and instead only “heard” hard metrics I could see. It’s the classic Goodhart Law, I did not expect it to apply to my body and mind.³

One of these signs was reflux. I started having reflux problems about a year and a half before the anxiety attack. It was frustrating, but not debilitating. I ran experiments to figure out what foods trigger it, broken down by time of day, and over time got a much deeper understanding of my body. However, this was treating the symptoms, not the cause. Reflux became much rarer after I quit.⁴ I’ve talked so much about systems but failed to see this connection in my own system.

Another sign was the weight gain. This graph was going up and to the right - and this is one of those graphs you definitely do not want on a hockey stick trajectory.

The data collection was manual here. I was inputting these numbers every day. I was seeing the graph every week. The alarm bells still failed to trigger, or when they did trigger, they were silenced. “It’s fine, I’ll start working out again when I have more time”. “Kicking myself over it doesn’t help me feel better”. “I already have enough on my plate”.

Present me would treat this a lot more seriously than past me. The inner monologue above suggests things have gone very much out of balance. It’s a stool missing one of its legs.

A third wedge that further complicated the above was this iron deficiency I didn’t know about. Borderline iron levels don’t get flagged on blood tests, but can cause fatigue and post-workout dizziness. Thanks to the NHS doc who noticed it after several others glossed over it.⁵

This also made my anxiety much worse - and for a while I was wrongly attributing the tiredness to anxiety, and turtled in quite a bit.

But still, I’m glad it came crashing down in 2024 rather than in 2028. The faster I understand, the faster I can course correct, and that’s exactly what 2025 has been about.

Rebuilding

I quit my job in January, after 6 months of trying to figure it out while keeping the job. I couldn’t bring myself to work anymore, and it was better to stop than drag it out. It wouldn’t be fair to my colleagues, and I’d feel much worse about coasting and making excuses too.

What followed was a period of lots of joy, new routines, new hobbies, and pushing my limited self to feel whole again. I discovered a love for board games, made lots of new friends, played a lot of tennis and pickleball, which made me realise how much I was missing out on.

It was slow though. Anytime I got impatient and pushed too hard, the anxiety would creep back in. This was.. weird. I was so used to pushing ahead that slowing down felt awkward. I was impatient to recover. Over repeated instances, I learned to calm down.

What feels like “pushing” can be true for even the smallest of things. For example, in the early days, I “pushed” myself when I went out to watch a movie with a few friends. It felt miserable. I felt trapped in my seat, watching the gore in Deadpool. This made me feel worse - “Why am I feeling like this? These are all things I used to enjoy?”. It spirals so quickly when I’m not noticing. I dipped to the bathroom to catch my breath, and went on a walk after to calm down. I decided that was enough adventure for the week, and turtled back at home.

I wouldn’t have gotten out of my new shell at all, had I not pushed; but I also couldn’t push too hard. It was important to take things slow, but to also test the edges of my capabilities. It’s like running evals for an LLM - we do it because we don’t have a good intrinsic understanding of AI model capabilities. I didn’t have a good intrinsic understanding of my limited self’s capabilities, so I ran my own evals. The difference is I’m not static. The act of testing my capabilities extends them, and sometimes shrinks them.⁶

However, my body doesn’t always align with what my mind wants. There’s always some apprehension in the body - call it anxiety, call it nervousness, call it jitters, call it excitement.⁷

Over time, this apprehension doesn’t go away. I have just gotten better at dealing with it. Enough examples of going out and having fun reinforces this idea that there’s nothing scary.

I still close off sometimes though, and that’s okay. Sometimes I just want to be comfy, and so I be comfy.

What I’ve learned

Looking back, a lot has changed in the last 5 years. The overarching theme is that theory sounds nice, but reality is messy. 2019-me was directionally correct. I would be in a much worse spot today if I didn’t think through life the way I did and lay down my principles.

Reality has a surprising amount of detail, and systems need reinforcement. Knowing what was happening and why it was happening wasn’t always enough to make a change. I needed the lesson drilled into me.

I felt the same about my second year of work review coming across as overconfident.⁸ And yeah, it was. But I still stand by it. If I don’t see myself as who I want to be, it’s much harder to reach where I want to be.

Environment > willpower

Structural design wins. I’ve yet to see success with willpower based resolutions. Maybe I’m weak-willed, but I’ve learned to stick to what works best for me.

My internal monologue still goes at it wrongly - “I need to go to bed early”, “Don’t eat greasy food late, it will give you sleep problems”, but what helps the most is designing my life such that this happens by default.

An 8 AM alarm every day means I’m usually too tired to stay up late, and works wonders for my sleep. It’s not the whole picture though. Everything is connected, and if I’m too stressed at work, stress-eating late at night, and then revenge bedtime procrastinating⁹ - then I invariably end up sleeping poorly. Fixing this doesn’t mean fixing sleep, it means fixing whatever else is broken, and sleep will fall back into place.

Every lasting behaviour change I’ve made has been environmental, not motivational. It was magical how many of these problems went away once I quit.

No junk food at home does wonders for healthier eating. SAD lamps make winters happier. Cold, dark room leads to much better sleep. Good sleep instantly elevates the day. Moving houses made me realise how important sunlight is to me. A living room full of sunshine sparks so much joy. As do regular weekly tennis sessions.

I can change things to be the way I want them. I can ask more of the world than the default.

Stress reveals itself through control

When I can’t wind down after work, when I am always on - that’s a sign of stress. That’s a sign I can’t shut off. That’s a sign I need to change things.

When I’m stressed, I grip tighter. I want to fix things so they stop being a problem. I feel the stress will go away when the problem has gone away. I keep thinking about the problems no matter what I’m doing, which causes more stress. This is counterproductive.

I’ve learned to trust myself, to let go. This is where high observation becomes somewhat of a burden. Tracking metrics when they’re all going poorly causes a bit more extra stress. Which in turn causes me to grip a bit tighter.

Sometimes, I need to walk away from the environment that incubates this stress. That’s why I quit my job.

The same patterns repeat

This is surprising. My first work review was very cerebral - things I’d learned, how to test code, frameworks to think about code design etc. etc. My second work review was a lot more.. emotional. Looking at the soft skills, how it feels to take on a big new challenge, the fear of failure, etc. etc. The loop was much faster - these posts came out over two years.

Now, this same pattern is repeating over six years of life reviews. The first post was all about frameworks on how I want to design my life, the next few years were about implementing it and seeing them fail. And this post is the culmination of those patterns. Some did well, but what became more important is how I felt, how I overcame challenges, not just intellectually, but by teaching myself to listen to my body, and being comfortable with anxiety. When I embraced it, felt it for what it was, and repeatedly felt that experience over months, anxiety the “bad thing” finally went away.

These same patterns show up at the micro level too. I have to rediscover the same things again and again for it to stick. I’m very much like an LLM in this sense, I need to experience a lot more examples than one would expect before it sinks in.¹⁰ Especially when it comes to complicated, non-intuitive situations, or where intuition leads me astray.

I sacrifice stability for growth

…and I still like this a lot. Sure, it leads to problems, and instability over time, but I’d do it all over again. Would I make better decisions given I know what I know now? 100%. Would I refrain from jumping into the next big thing worrying over whether I’ll fall in the same loop again? Nope. I might still flounder in a different way, but as long as I learn from it, as long as I grow, that sets me up for a more successful third loop.

It’s a trade off I’m very willing to make. Compared to 2019-me, I’m more resilient, a little wiser, and most importantly, more content.

Four legs on the ground again, it’s time for the next adventure!

In micro-gravity, it might take very long. ↩
And maybe it’s not a stool, but an N-dimensional object, which requires N+1 legs to be stable, and I don’t realise these other legs exist because they’ve never been absent in my life for long. Or maybe just like in a Principal Component Analysis, the first four dimensions account for majority of the stability and the others don’t affect things that much. Or maybe I should stop stretching this analogy so much. ↩
“When a measure becomes a target, it ceases to be a good measure”. This happens in organisations all the time. Inspired by Peter Drucker’s timeless wisdom, “What gets measured gets managed”, the quest to measure metrics begins, as it did for me. However, what’s easy to measure isn’t always what determines overall health. For example, weight increase is good if I’m gaining muscle mass, but bad when I’m gaining fat. Weight is much easier to measure though than muscle mass, so 🤷‍♂️. ↩
Why not gone completely once the stress is gone? Again, the body is a complex adaptive system, and once the gut health is fucked, it seems like it takes a lot longer for it to go back to normal. ↩
And a lesson for me to run Claude / ChatGPT over my blood reports. I don’t know if the 2023 models would’ve caught it, but the early 2025 one definitely did, and managed to connect with the slightly off values in other parameters. And these models are only getting better 🤯. ↩
When I push too hard. ↩
I’m glossing over this right now, but the way I frame it makes a huuuge difference! My relationship with anxiety is a lot less negative now than it was initially, and I imagine for a lot of people anxiety is a “bad” word. So when you frame the jitters as anxiety, it seems like a bad thing is happening to you, which leads to avoiding it. I’ll maybe talk more about this in a separate post - let me know if you’d like to hear more about it. ↩
and it got trashed on Hacker News for the same, as expected 😂 ↩
“Learned a very relatable term today: ‘報復性熬夜’ (revenge bedtime procrastination), a phenomenon in which people who don’t have much control over their daytime life refuse to sleep early in order to regain some sense of freedom during late night hours.” - tweet deleted now, and I don’t want to link to news slop. ↩
And this is the third time I’ve referenced this idea of repetition to make things sink in, in a different way. Co-incidence? I think not. ↩

How to setup duration based profiling in Sentry

2023-10-04T00:00:00+00:00

I recently came across a gnarly issue: Every morning we’d see a latency spike in our API for for about 7-8 minutes. More importantly, it happened for a small fraction of requests (0.1%), making it pretty hard to reproduce.

I figured leveraging Sentry’s profiling feature would be a good way to catch this. But this has its own problems.

The first problem is that it’s very expensive to profile all requests: a billion samples where most of them are useless is a waste of money.

Sentry allows sampling requests, which somewhat takes care of the cost problem.

The second problem is that Sentry by default samples based on the count of events. So if you have a sampling rate of 1%, you’ll get approximately 1 in 100 requests. But, since most transactions are pretty fast, these are still useless. And 10 million requests instead of a billion is still a lot of wasted money.

I required a way to sample requests by duration: instead of sampling by the request ID, I want to sample by duration. Give me 50% of all requests that took more than 2 seconds, and 0.0001% of all requests that took less than 2 seconds.

The magic of this kind of sampling is that it gives me the most important spans, while being much cheaper than the default sampling.

The docs don’t mention this is possible at all, but combing through the source code, I found a way to do this. Here’s how it works:

Sentry has a before_send_transaction function that fires before sending a transaction to Sentry. We can use this to modify the transaction before it’s sent, or add any additional logic (like sampling by duration) to drop the transaction.

This is a pure function that takes in the transaction, and either returns a transaction or None. If it returns None, the transaction is dropped. So, any kind of logic depends on what’s inside the transaction. See the appendix below for a sample transaction.

In my case, all I need to know is the request URL (so I can apply this only to the API request I’m interested in), and the start and end timestamp (so I can sample based on that).

Note: The only gotcha to remember here is that you need to manually sample things, instead of using the traces_sampler or any builtin sentry sampling, since those apply before before_send_transaction, which means it’s too late by the time you get the transaction.

Here’s the code I ended up using:

def before_send_transaction(event, hint):
    # get the request URL
    url_string = event.get("request", {}).get("url")
    # only apply this to the request I'm interested in
    if url_string and "decide" in url_string:
        # default sampling rate based on all requests.
        DECIDE_SAMPLE_RATE = 0.00001  # 0.001%
        should_sample = random() < DECIDE_SAMPLE_RATE

        transaction_start_time = event.get("start_timestamp")
        transaction_end_time = event.get("timestamp")
        if transaction_start_time and transaction_end_time:
            try:
                parsed_start_time = parser.parse(transaction_start_time)
                parsed_end_time = parser.parse(transaction_end_time)

                duration = parsed_end_time - parsed_start_time

                if duration >= timedelta(seconds=8):
                    # return all events for transactions that took more than 8 seconds
                    return event
                elif duration > timedelta(seconds=2):
                    # very high sample rate for transactions that took more than 2 seconds
                    return event if random() < 0.5 else None

            except Exception:
                pass
        return event if should_sample else None
    else:
        return event


sentry_sdk.init(
    # ...
    before_send_transaction=before_send_transaction,
)

And finally, here’s a real world code example of how we did this at PostHog.

So, what ended up being the issue? I’m not sure yet, but I have these lovely spans to help me investigate

Slow Sentry spans

Appendix: Sample transaction

Here’s a sample transaction, which helps us determine all possible fields we can use:

{
    "type": "transaction",
    "transaction": "/decide/[#].*",
    "transaction_info": {
      "source": "route"
    },
    "contexts": {
      "trace": {
        "trace_id": "e9c1XXXXXXXXXXXXXXXde238",
        "span_id": "89XXXXXXXXXXXXXcc",
        "parent_span_id": "90XXXXXXXe99",
        "op": "http.server",
        "description": null,
        "status": "ok",
        "tracestate": "sentry=<base64 encoding of the above info>",
        "dynamic_sampling_context": {
          "trace_id": "e9XXXXXXXXXXXXXXXXXXXX8",
          "public_key": "3XXXXXXXXXXXXXXXXXXXXXXXXa",
          "sample_rate": "1e-05",
          "release": "4XXXXXXXXXXXXXXXXXXXXXXXXXXXXX14",
          "environment": "production",
          "transaction": "/api/projects/{parent_lookup_team_id}/persons/"
        }
      },
      "runtime": {
        "name": "CPython",
        "version": "3.10.10",
        "build": "3.10.10 (main, Mar  7 2023, 14:21:53) [Clang 13.1.6 (clang-1316.0.21.2.5)]"
      }
    },
    "tags": {
      "http.status_code": "200",
      "origin": "unknown",
      "referer": "unknown",
      "library.version": "unknown"
    },
    "timestamp": "2023-10-03T09:47:17.681238Z",
    "start_timestamp": "2023-10-03T09:47:13.542852Z",
    "spans": [
      // A list of child spans in the transaction
      {
        "trace_id": "e9XXXXXXXXXXXXXXXXXXXX8",
        "span_id": "e9XXXXXXXXXXXXXXXXXXXX8",
        "parent_span_id": "e9XXXXXXXXXXXXXXXXXXXX8",
        "same_process_as_parent": true,
        "op": "event.django",
        "description": "django.db.close_old_connections",
        "start_timestamp": "2023-10-03T09:47:13.543012Z",
        "timestamp": "2023-10-03T09:47:13.543020Z",
        "data": {
          "signal": "django.db.close_old_connections"
        }
      },
      {
        "trace_id": "e9XXXXXXXXXXXXXXXXXXXX8",
        "span_id": "e9XXXXXXXXXXXXXXXXXXXX8",
        "parent_span_id": "e9XXXXXXXXXXXXXXXXXXXX8",
        "same_process_as_parent": true,
        "op": "middleware.django",
        "description": "posthog.middleware.PrometheusBeforeMiddlewareWithTeamIds.__call__",
        "start_timestamp": "2023-10-03T09:47:13.543162Z",
        "timestamp": "2023-10-03T09:47:17.681195Z",
        "tags": {
          "django.function_name": "django.utils.deprecation.MiddlewareMixin.__call__",
          "django.middleware_name": "posthog.middleware.PrometheusBeforeMiddlewareWithTeamIds"
        }
      },
      {
        "trace_id": "e9XXXXXXXXXXXXXXXXXXXX8",
        "span_id": "e9XXXXXXXXXXXXXXXXXXXX8",
        "parent_span_id": "e9XXXXXXXXXXXXXXXXXXXX8",
        "same_process_as_parent": true,
        "op": "db",
        "description": "SELECT \"posthog_persondistinctid\".\"person_id\", \"posthog_persondistinctid\".\"distinct_id\" FROM 
        \"posthog_persondistinctid\" WHERE (\"posthog_persondistinctid\".\"distinct_id\" IN (%s) AND \"posthog_persondistinctid\".
        \"team_id\" = %s)",
        "start_timestamp": "2023-10-03T09:47:17.573475Z",
        "timestamp": "2023-10-03T09:47:17.583230Z"
      },
    ],
    "event_id": "e9XXXXXXXXXXXXXXXXXXXX8",
    "extra": {
      "sys.argv": [
        "manage.py",
        "runserver"
      ]
    },
    "request": {
      "url": "http://localhost:8000/decide/",
      "query_string": "v=3",
      "method": "POST",
      "env": {
        "SERVER_NAME": "1.0.0.127.in-addr.arpa",
        "SERVER_PORT": "8000"
      },
      "headers": {
        "Content-Length": "282",
        "Content-Type": "application/json",
        "Host": "localhost:8000",
        "Sentry-Trace": "e9c121e003284df18bf9905ad6dde238-90219bf565efce99-0",
        "User-Agent": "posthog-python/3.0.1",
        "Accept-Encoding": "gzip, deflate, br",
        "Accept": "*/*",
        "Connection": "keep-alive"
      },
      "data": ""
    },
    "release": "e9XXXXXXXXXXXXXXXXXXXX8",
    "environment": "production",
    "server_name": "192.168.1.104",
    "sdk": {
      "name": "sentry.python.django",
      "version": "1.14.0",
      "packages": [
        {
          "name": "pypi:sentry-sdk",
          "version": "1.14.0"
        }
      ],
      "integrations": [
        "aiohttp",
        "argv",
        "atexit",
        "boto3",
        "celery",
        "dedupe",
        "django",
        "excepthook",
        "logging",
        "modules",
        "redis",
        "stdlib",
        "threading"
      ]
    },
    "platform": "python",
    "_meta": {
      "request": {
        "data": {
          "": {
            "rem": [
              [
                "!config",
                "x"
              ]
            ]
          }
        }
      }
    }
  }

How to simulate a broken database connection for testing in Django

2023-01-15T00:00:00+00:00

I spent a very long amount of time trying to figure out how to simulate a broken database connection in Django. The problem is that not only do you want raw cursors to timeout, but also all models accessing the database.

This means, wherever we call Model.objects.filter(), Model.objects.all(), or connection.cursor(), the operation should fail.

Google and ChatGPT both produced very disappointing results, which forced me to dive into Django’s source code and think of creative solutions to the problem.

As a result, I think people will find this useful, future-me will be able to google the right answers, and a future-ChatGPT will be able to give the correct answer. Let me know if you find another way!

Option 1: Patch raw cursors

If you’re directly using a raw cursor, you can do the following:

from django.db.utils import OperationalError

with patch('django.db.connection') as mock_connection:
    mock_connection.cursor.side_effect = OperationalError('Connection timed out')

    # write test that requires a database failure

The problem with this option is that this doesn’t work everywhere. It’s also not feasible to patch all your models, so this can be brittle.

Option 2: Patch Django query internals

An approach that works for all models is to patch the internals:

from django.db.utils import OperationalError

with patch('django.db.models.sql.compiler.SQLCompiler') as mock_compiler:
    mock_compiler.side_effect = OperationalError('Connection timed out')

    # write test that requires a database failure

The SQLCompiler is instantiated whenever a model tries to access the database. This ensures that whenever you call something like Model.objects.all(), this side effect is triggered.

The disadvantage of this method is that it relies on django internals which can change over time. So, on a random upgrade, you might find that your tests now fail.

Greatest Option 3: Leverage database wrappers

The best way I’ve found to simulate a broken database connection is to leverage database wrappers. This is at a higher level of abstraction than the internals, and thus a lot less brittle.

Django provides a context manager to wrap all database query executions. You can do anything in here. Their main recommendation is to use this for instrumentation, but anything goes, and it’s ideal for testing.

from django.db.utils import OperationalError

class QueryTimeoutWrapper:

    def __call__(self, execute, *args, **kwargs):

        raise OperationalError("Connection timed out")
        
        # If you want the query to run, you'd use the below line
        # return execute(*args, **kwargs)

with connection.execute_wrapper(QueryTimeoutWrapper()):
    # write test that requires a database failure

Read the docs for more information.

For a real world example, check out this PR in PostHog, where I leveraged this technique to write tests.

Why is this useful?

When I find it takes me longer than an hour to find a solution, I usually try to think of different approaches to side-step the issue. In this case, I couldn’t do without simulating a broken database.

I was trying to write defensive code, ensuring we don’t return 500 errors even when the database is down, by clever use of exception handling and caching.

In this case, there’s several random database models that might be called, and I can’t keep track of them. If I miss even one, things will blow up. So, testing my changes work for all cases is important. Further, it’s a good test for knowing whether caching is working as intended, as thanks to the patching above, any time a function uses the database instead of the cache, it will blow up and raise the OperationalError.

Thanks to Karl for pointing me to Option 3.

The “People fuck up because they’re not like me” Fallacy

2022-11-13T00:00:00+00:00

Recently, I’ve noticed an interesting failure mode in humans. Here are a few examples, can you tell what’s wrong?

I eat meat. My family doesn’t. At our last family gathering, my family urged me to go to the dentist, because it seemed like my teeth were sharper. The only reasonable conclusion was it’s an effect of meat, like how carnivores have sharp canines.

I have a friend who washes his hands regularly. Like, every three hours. He also has a severe dry hands problem. His family is convinced it’s because he washes hands too often. The thinking goes something like: “We don’t wash hands regularly, and we don’t have dry hands. He washes hands too often, and he does have dry hands!”

This is an interesting case of privileging the hypothesis, where you figure an obvious cause for people’s problems is doing things differently.

The way I see others as different from me, is the way I expect their ill-effects to present themselves.

Further, it’s not just about trivial stuff.

I was debugging code with a colleague once, and the first thing I jumped at as problematic were things done differently. Regardless of what the compiler says is the problem. I didn’t feel comfortable with things done in such a way, and suspected them to be the root cause, just because I don’t code in the same way.

The way I see things done differently from me, is the way I expect inefficiencies and problems to arise.

It’s not only personal differences that show the same way, but cultural differences too. The same people make up a culture, so a lot of inter-personal problems apply to inter-cultural problems as well.

In the 14th century, the bubonic plague struck. Perhaps surprisingly, the Jews were relatively less affected. They didn’t use common wells, washed hands regularly, and washed their dead before burials. Around the same time, anti-semitism was widespread. Other people used to drink from the well, not wash their hands, and were healthy, until they were not. Seeing the antics of the Jews, and their relative continued health convinced people that the Jews were responsible for the plague. Sadly, this lead to widespread massacares.

As we know now, Jews were definitely not responsible for the plague. It was rats instead.

The way I see other cultures as different from mine, is the way I expect their ill effects to shine.

In all examples I’ve shared above, we let the contrast between personalities and cultures govern what’s wrong with other people. It’s an interesting mish-mash of two fallacies:

Contrast bias occurs whenever you’re comparing two or more things. The differences between these two things take up all your attention, while viewing them individually you might not reach the same conclusions.

For example, a person will appear more attractive than they do in isolation when simultaneously compared to a less attractive person.

For the same reason, I’d jump to conclusions around dry hands and washing hands, simply because I’m contrasting the two, rather than thinking clearly. Can more wetness lead to dryness? Actually, yes, it can, so the heuristic isn’t always wrong. Rather, it’s about discarding all other information because the contrast pops at you.

The typical mind fallacy occurs whenever you assume that every person’s brain is similar to yours. If you like rock music, everyone who doesn’t is plain weird. You compare people, and subject them to the same goals and ideals as yourself. A perverse way this shows up is thinking others are ‘lesser’, when they can’t do what you can, specially when you assume they’ve had the same opportunities as you.

While most people have similar goals, their values and constraints are usually different. It’s hard to tell a recessive gene is the cause of resisting the plague, when you can’t see the gene in action. It’s easier to contrast and jump to conclusions that make sense from your mind’s point of view.

With most biases, the trick is to notice what’s happening, have a name to call it, and you’re in a good spot to at least know when you’ve already fallen for it. So, to end, I’ll leave you with a name. It’s the “People fuck up because they’re not like me” Fallacy.

How I Own Projects as a Software Engineer

2021-12-14T00:00:00+00:00

When you take ownership of a project, something changes. You become responsible for everything that happens with the project.

Things go wrong? That’s on you. Customers are unhappy? That’s on you. You shipped something that doesn’t solve the problem? That’s on you.

Ideally, you have a process that reduces the number of ways things can go wrong.

I started at PostHog ~8 months ago. Since then, I’ve been fortunate to own 2 big projects. I’m nowhere near experienced enough to be talking about prescriptions, but that’s not the point. A big part of growing quickly is understanding what you’re doing and why you’re doing it. So, this post is my model of how to own and build new products.

Something magical happens when you own the entire stack. From talking to customers, to deciding what to build, to deciding who will build what, to finally getting feedback, and then redoing this cycle. You see the entire flow, which gives you better context on what to do. Sometimes, software isn’t the way to solve problems.

Broadly, I’ve noticed 5 steps. The goal behind each of these is to increase your chances of success. I’ll explain them, and follow up with metas behind these steps. Finally, since PostHog is open-source, I’ll link to precise real-world examples for what I’m talking about.

Gather Context

Trying to find solutions before you understand the problem will lead you around in circles. First, understand the problem. This begins with understanding the use case.

An interesting project I recently worked on was Experimentation. If you think of it in terms of what you should be building, it would be: “Build an A/B testing platform”. This problem severely lacks context. When you think in terms of what to do, you lose track of what users really wanted.

It also makes it harder to prioritise & figure out what to build. For example, say you’re just starting off with this project. What do you include in the MVP? What are the risky things you need to test quickly? You can’t answer these questions because they’re outside the scope of the problem.

Things are different when you think in terms of the use case. Begin with: “People have trouble A/B Testing because they are missing instrumentation, and the results they get are hard to understand.” This tells you what you need to do: (1) Make instrumentation easy & (2) Make results easy to understand.

Getting to this point is hard, and the first obstacle to building a successful product. To understand the use case, you need to talk to users. Ask questions. Go deep into what they say. Here’s how PostHog does it.

Once you understand the use case, you can judge what problems are related to your own, and whether existing solutions work or not. In a world where similar problems have been solved before, it would be a shame to not understand how others have solved these problems.

Take some time to see how others have solved these problems. With A/B testing platforms, there’s hundreds of them in the wild! What helped me lead this project better was to know how these A/B testing platforms work, which problems they focus on solving, and take inspiration from it.

For example, I realised that most A/B Testing platforms are standalone: they’re unbundled, so instrumentation and deciding on target metrics happens elsewhere (say, your analytics suite), and you come here for doing analysis on the results. Since PostHog already has Feature Flags for instrumentation, and it’s main job is literally an analytics suite, there’s real opportunity for bundling here. This is what governed our thinking about this feature.

Looking at solutions also gives you a better understanding of the risks you might be taking. With experimentation, since our goal was to make results easy to understand, and reduce the number of things users have to know to run a good A/B Test, we decided to test out the Bayesian analysis route. This gets rid of concepts like p-values, statistical significance, confidence intervals, and focuses on a singular value: the probability that your variant is better than the control. For the nitty gritty details, see here

Figure out a solution

Once you have context, you can start thinking of solutions.

One trap I’ve noticed myself falling into is taking inspiration too heavily from existing solutions. How can you build a better version if you repeat the same mistakes they’ve made?

Another trap is thinking only in terms of what’s possible technically. This is specially a problem with engineers. You know how the system works, you know the compromises you’ve made, so you limit the world of possible solutions that fit into these existing constraints. This is a problem, because it often underestimates what’s possible.

The way I evade both these traps is to start with ideal solutions. What solution would completely solve for the usecase? Doing this allows me to tease out the first principles. “What is it about the ideal solution that makes it ideal? What is the crux that solves the problem?”.

For example, with experimentation, I came up with 3 first principles born out of an ideal solution:

The numbers have to be 100% accurate. No estimation, no rough calculation, no infrastructure affecting the A & B test buckets. Someone who was in bucket A can’t move to bucket B two days later.
The results should be easy to understand
- There’s a lot more leeway here. Figuring this out well means going back to your users and asking for feedback.
People should be able to test their changes before launching the A/B test.
- The experiment becomes invalid if both A & B variant are showing the same thing when they shouldn’t be.

What’s very cool about engineers owning products is the conversation you can have with yourself. On one side are the technical constraints: what’s possible given what we have now? On the other side are the use cases: Disregarding all technical constraints, what does the ideal solution look like?

When technical constraints clash against the ideal, I can go back to the first principles. Now, given I want to preserve these principles, how can I solve this problem?

The above question forces me to think of levers I can pull. I often believe that the only lever I have is the technical constraints: What clever optimisations can I do that will make this possible?

But this isn’t true. There’s another powerful lever I have: solving the problem differently. In practice, this means changing the UX, while preserving the first principles.

For example, I clashed against a hard technical constraint in experimentation. It’s not something that can’t be solved, but doing it takes way too much engineering resources. Instead, I opted to change the UX. We’re still respecting the three first principles, it’s just that the creation flow is different from how I expected the ideal one to be.

Worth noting here again are the three principles for experimentation. While (1) is technical, there’s a lot more flexibility with (2) and (3). The UX lever becomes a lot more potent when you hit technical constraints you can’t overcome. That’s exactly what I did. Principle (3) doesn’t prescribe anything about existing Feature Flags. So, to overcome the technical constraint, we added a new UX constraint: People can’t reuse existing feature flags.

Note:
Next time I’m leading a project, I should make the principles explicit. They stay in my head most of the time, but it ought to be very helpful for the team to know them as well: it’s valuable context that helps everyone make decisions.

Build

We’ve now figured out the principles thanks to the usecase we’re solving for, and have a solution in mind. I want to move the quickest when building.

As the owner, my biggest responsibility here is setting priorities. Priorities help everyone move quickly and get shit done. I’ve been heavily influenced by Scott Berkun’s post on making things happen.

This means creating an ordered list of tasks that need to get done to achieve your goals. For huge projects, I might use a Kanban board, but for smaller ones, a single GitHub issue seems ideal. For example, here’s one I created for Experimentation.

In most cases, I get a MVP out first. Choosing what parts are important needs the right priorities set. Does it need proper UIs or not? Can it work with mock data? What’s the smallest working expression of our first principles?

The first project I led, I faced this whirlwind of tasks when building: there’s so many fucking things to do: a few people who need my help with stuff, figuring out designs, doing PR reviews etc. etc. A few days into the implementation had me juggling between all of these. I was moving around a lot, without getting much done. The core problem, I realised, was that I’d jump on whatever was dropped on my plate.

If I’m coding something up, and a PR request comes in: since no one else will be reviewing this, and I’m the owner - I’d want to unblock others quickly so they can keep moving fast - I’d drop what I was doing, do the review first, and then get back on whatever I dropped. Like a stack, last in first out.

This is terrible prioritisation. I had read Scott’s blogpost a while ago, but feeling it viscerally was what made it click for me: I need to get my priorities in order & follow through on them. This resolved things much better. I got rid of the thrashing problem by using a better scheduling algorithm. Whatever is the highest priority gets done first, no matter what comes in.

This ties back into how important the first two steps are. If you skipped them before building, your goal would simply be “build an A/B Test platform”. You’d have a much harder time prioritising because you lack context, and don’t know principles you can’t compromise on.

Being effective here also means keeping people on the same page, and ensuring that they can be productive. With more than a few people, I imagine this can get hard. I’ve only led very small teams (1-3 people), so things have been smooth so far.

I also appreciate my manager’s job a lot more now. I need to think about what every person does, and how they work. Given how the Big 5 Personality test accurately represents people in 5 dimensions, I’d hypothesize most people are categorizable. There’s only a few dimensions to them.¹

Like a Fourier Transform, instead of working with those 5 dimensions, it’s more effective to work in dimensions that are easier to measure at work. I’m yet to figure these out completely (too few experiences dealing with people), but a few dimensions that stand out:

Do they ask for help if they get stuck.
To be productive, do they need me to (a) just set context; or (b) micromanage.
What are their strengths?
Are they gear-driven or behaviour-driven?

I’m very gearsy - which means when I propose UX designs, I tend to go for “transparent internals” approach. As I’ve come to see, this is almost always the wrong thing to do. Most users don’t care about how their problems are solved, just that there’s a button which solves them.²
????

I think what makes management even easier at smaller companies is that most people are around the same on these dimensions, because the interview process selects for this. For example, at PostHog, once context is set, you can let people do what they feel is the most important.

I appreciate how I’m a complete n00b at management. But then again, I don’t think just adding years is going to make me better at it. Like I said in the introduction, if I don’t understand what I’m doing and why I’m doing it, I can’t improve quickly. Hence, my hypothesis on management.

Hopefully, you’re effective enough that you and your teammates get an MVP out. The next step is gathering feedback.

Gather Feedback

The best way I’ve found so far is doing user interviews. Nothing compares to the high fidelity one on one feedback, where you can go deep into whatever they say.

When gathering feedback, there’s two important things you’re testing:

Whether the principles you arrived at are indeed the first principles.

It’s easy to misunderstand what users really want. With a concrete MVP in place, you could ask hypotheticals like: “How would you do <Important Thing> if this feature didn’t exist?”. You’re looking for confirmation that the information you gathered in the first step is still valid.

For example, with experimentation, I would go deep into what users mean by the results being easy to understand. Is us telling them that A has a 85% chance to beat B good enough? Do they need more information? Why?
Whether your solution is a reasonable expression of those first principles.

Don’t just confirm you’re on the right track; ensure you actually solve the problems. Paolo, PostHog’s Product Manager, does this by giving users challenges. For example, “Say you want to create a new experiment. How will you do it?”. I often follow up with questions, like ‘What do you think this graph means?’ or ‘Why are you getting that warning?’

After these challenges, it’s a good idea to ask for general feedback: What’s missing? What do you love? Did this solve your problem?

At the end, you want to come out with two things: (1) Do users love this? and (2) If not, what’s missing? What do you need?

Align metrics with feedback

Just having happy users in feedback calls isn’t enough. I always need to ensure that metrics line up with what users are saying.

I like to go through two cycles of Gather context -> Figure out solution -> build -> gather feedback. At this point, things are usually good to go. If I’m doing things right, the second cycle is much faster: it’s adding on functionality that we skipped in the MVP, so the gathering context & figuring out solutions steps can mostly be skipped.

At this stage I’ll add analytics support (i.e. start sending in events about how users are using the product), start releasing to a % of the world, and see how things go. You ought to be able to answer questions like: “How many people are using it?”, “Of the people using it, how many find it valuable?”, and any other relevant product questions.

I build a dashboard of metrics for every product I’ve worked on, and check it every Friday. This is important in the early weeks right after you launch, since it shows you if there are bugs with event capture, and if you can answer all questions you have about the product. If I can’t tell which part of this product is most popular, I’m probably lacking event instrumentation: I should add those missing events as soon as possible. Otherwise, I’m flying blind.

Usually, it takes longer for patterns to show up in metrics. At this point, I’ll put the project in background mode, wait for user patterns to show up, and then decide what I want to do about it.

For example, with my first ever project, correlation analysis, our usage funnel today looks something like this:

People love correlation analysis a lot, as evident by the 85% conversion rate for the second step. The problem is discovery: most people don’t know it exists. So, whenever I come back to it next, the problem I’d want to solve is making it more discoverable.

Meta: Create feedback loops

A common theme across all the steps I want to highlight is feedback loops. To work quickly & to make sure you’re doing the right things, you need several feedback loops. Both, long term and short term.

Each step above has its own quick feedback loops, where you get feedback within hours.

For example, when gathering context, don’t do it in a silo. Do it with your teammates (specially product owners). Explore competitor products together. Make the thread open, so everyone in the company can see and contribute to it. It’s valuable to hear from colleagues who might have more context because they were an insider.

In the building phase, fast feedback loops mean small pull requests and quick reviews! Imagine how much faster you get things done and how much better your code looks when you have small PRs that get reviewed quickly, vs. a 500 line change that takes ages to get reviewed.

Apart from these quick feedback loops, there are slower but still important feedback loops between consecutive steps. The stage outputs aren’t set in stone. There’s going to be times when things come up that you didn’t think about earlier.

New technical constraints might show up while implementing the solution, that forces you to go back to the “figuring out the solution” stage. Don’t get married to the initial idea you came up with. Divorces are hard.

While working on experimentation, we decided to allow reusing an existing Feature flag to do experiments. This made sense because people would create the feature flag, test the A & B versions look alright, and then use that same feature flag in an experiment, without having to do any code changes.

However, during implementation I realised this clashed with one of the principles. Making things work like this meant the results would not be 100% accurate.³ So, as I mentioned above, we came up with the extra constraint of not re-using feature flags, and generating code snippets to explain how to test with this new feature flag.

Finally, the longest feedback loops come with usage over time. This means tracking your metrics properly. The final step, “aligning metrics with feedback” is precisely this: a long feedback loop that gets rid of the noise and shows you exactly how users are using this product.

The meta is to always create feedback loops.

Meta: Manage your Emotions

When I led my first project, something switched. For the first time, I felt this responsibility of getting shit done myself - there’s no one else I can fall back on. This led to me being a lot more “switched on”. I found it hard to switch off after work. I worried about whether I’ll hit my self-imposed deadlines, about technical challenges we haven’t yet solved, and whether I’ll succeed or not.

I love switching off after work. This is wonderful for my mental health, and allows me to freely enjoy my other hobbies outside of work. So, I knew that owning projects where I’m always switched on like this wasn’t feasible.

This was all on me: I wasn’t used to leading projects, and that took a toll on me.

The next time around, things were a lot more stable. I don’t know what changed, but I felt a lot more in control. I think familiarity with the process helped me face this anxiety better. Not surprisingly, I’ve faced something like this every time I’m trying something new. It’s a feeling I’ve come to accept.

Another thing I noticed was that anxiety towards the end of a project shot up. The MVP felt easy: there’s no polishing required, it’s just the concept, get it out and you’re good. The project isn’t done with the MVP though. To finish, you actually need to tie up all the loose ends and polish the UI. The closer I got to finishing, the more I worried. I think a big part of this was that I had never built a pretty frontend before (every frontend I built before looked like shit). It was an unknown that I didn’t care about earlier, because I knew there’s atleast a few people at PostHog who are very experienced at building pretty things. Working through this unknown myself was very useful: it helped me face my “fear” and gain confidence in an area I was unfamiliar.

Since I was the project owner, it would’ve been just as easy to pass this work onto someone else. But noticing what I was doing and why I was doing it helped me make better decisions.

I don’t have much to offer in terms of what to do about emotions. They come, change your direction, and flitter away. What I’ve found helpful is noticing these emotions. Notice how you’re feeling and why you’re feeling that way. It gives me clarity, and just doing this is enough to make better decisions.

Conclusion

Each of the 5 steps above are crucial. They bring something important to the table, and I suspect are necessary for every successful product.

I figured the hard part of software engineering was building things. But now, maybe it’s figuring out what to build. Knowing what to build seems rarer. That’s surprising to me, since you don’t need to know how to code to figure out what to build.

There are probably more subtle things I’m doing wrong (and right), which I haven’t been noticing yet. But this post is my playbook: new things I notice get inserted into one of these steps, warnings get added, and I keep getting better and better.

Wrapped in here is the third meta: To improve quickly, you need to know what you’re doing and why you’re doing it.

It’s hard for me to believe that everyone is unique, etc. etc. Most people seem to have similar lives and the same problems. ↩
A bit of an overgeneralisation, depends a lot on the context/industry/solution. But since I’m too gearsy, phrasing it this way ensures I take it seriously. ↩
Not because the feature flag is already in use, but because of the type of feature flag. Real details here ↩

Funnels: The One Big Mental Model from Sales & Marketing

2021-08-22T00:00:00+00:00

The traditional funnel is a measurement device for, say, number of people converting in your product.

But funnels are a lot more universal, and you can use them to answer tough questions in Physics and Chemistry, not just for human behaviour. This post shows you how to use funnels more often, what are the general principles behind funnels, and finally, why this flexibility makes funnels the one big mental model from sales and marketing.

We’ll start with 4 weird examples of funnels in the wild.

Then, we’ll use the examples to decipher the general principles. We investigate what are the similarities and differences between the examples to come up with a general model for Funnels.

Overall, we’ll answer what exactly are funnels, why they’re cool, and where to apply them. Let’s dig in.

Crude Oil Distillation

To dissolve all connotations with existing usage of funnels, let’s start with something more arcane. Let’s go to Chemistry & fractional distillation.

Why do you think crude oil is called black gold? It’s not just because it produces petrol. It’s because it produces several kinds of fuels: gasoline, diesel fuel, heating oil, jet fuel, petrochemical feedstocks, waxes, lubricating oils, and asphalt. Crude oil starts with a mixture of these, and goes through several rounds of processing which spits these fuels out one by one.

At every round of processing, depending on the temperature, you get one product out of crude oil. So, for example, if you first set the temperature at 100 degrees, you get petrol/gasoline out of crude oil. Then, at 150 degrees, you get kerosene out of crude oil. And now the mixture has diesel oil, lubricating oil, and jet fuel left inside. You keep going through these steps, until you only have jet fuel left.

This is a funnel¹. You have a lot of “things” which are processed over different steps, such that you lose some, until you’re left with one thing.

Three more examples, and then we’re ready to draw the pattern out from it.

ETL Pipeline

Let’s jump to software engineering.

Say you have an excel sheet of survey results. You asked people for their names, work place, salary, and favourite color. Now, it’s time to process this data: First, you want to remove all excel rows that have a column empty. You dislike people who don’t fill in the full survey, and don’t care about their data. Then, since your favourite colour is blue, you’re only interested in people whose favourite colour is blue.

Finally, you have a set of people who filled the survey completely and like the colour blue, so you insert them into a new excel sheet.

This process is an Extract-Transform-Load, or ETL pipeline. You begin with a data source (excel), extract data from it, then transform it (remove nulls, choose colour blue), and then load it into another data store (excel again for our example).

The ETL pipeline is a funnel. You start with some rows of data, then you go through a series of processing steps which lead to discarding some rows, and finally, you end up with a smaller set of rows.

Of course, it’s possible that you don’t discard any rows. For example, if the survey happened in a diligent population who likes blue, everyone would’ve filled the entire survey, and chosen the color blue.

It’s also possible that you discard every row. If the survey happened in a red state, no one would say they like the colour blue.

Conversion Funnels

Now comes the text-book example from sales and marketing, the conversion funnel.

Say you have a website, and lots of people visit it.

The conversion funnel tracks how many users came to your website, how many of those who came signed up, and finally, how many of those who signed up started paying. It tracks how well people “convert” from being just a random visitor to someone paying you money.

Usually people drop off at every stage.

Below is a real funnel from the analytics company I work at, PostHog. Notice how you can get a lot of information from this. Most people do a pageview after signing up. That’s expected, and it’s quick - takes them 3 seconds to go from sign up to a pageview. The third step is a bit harder. It takes longer for people to reach, and not everyone is successful at it.

A Funnel created on PostHog.com

Pachinko

For our final example, let’s talk about the ball game Pachinko.

There’s lots of balls flowing through the machine, and only some specifc routes lead to winning. Each route for the ball is a funnel. What’s particularly interesting are the routes that lead to wins.

For creating a successful Pachinko machine, we’d want the winning funnels to have a small conversion rate. While as players, we’d want to choose the Pachinko machine with the highest conversion rate. Unfortunately for players, they don’t have enough data to figure out the conversion rates: it’s a closely guarded secret by Pachinko parlour owners.

Put together, all four examples are purposefully bizzare. They don’t seem quite the same, but they all use the concept of funnels. We can now look at the similarities and differences to figure out the core ideas.

Where Funnels make sense

Just like how the average of one value doesn’t make much sense², a funnel of one item doesn’t make sense. The power of averages, and funnels, come when you’re dealing with lots of things, and need a better way to summarize this data.

In the crude oil example, we were aggregating over molecules.
In the ETL pipeline, we were aggregating over rows of data.
In the conversion funnel, we were aggregating over people.
And finally, in the Pachinko machine, we were aggregating over balls.

This is a common property across all our funnel examples. Funnels only make sense with aggregates.

So, whenever you’re dealing with a large number of similar things, think if you can represent them as a funnel.

However, this isn’t enough. To create a funnel, you need two ingredients: the things to aggregate over, and the specific step definitions. For a good data summarization, you should choose good representative steps.

Take the Pachinko machine example: I glossed over how to choose steps. The specific pegs the ball touches aka the route of the ball makes sense because it’s actionable. It allows you to tweak the specific pegs to increase or decrease the conversion rate.

A bad step representation would be: ball inserted -> ball shot -> ball touches some peg -> ball enters a hole. This doesn’t help us manage the winning rate. However, it’s pretty useful for the person debugging the pachinko machine. If the conversion rate is not 100%, there’s something going wrong with the machine that engineers need to fix.

Choosing the wrong step definitions makes a funnel useless.

Funnels and Gears

We got our first principle by checking what’s common between the examples. Now, let’s reverse the question. How are the funnel examples different?

With crude oil, we know the precise chemistry. So, we know beforehand precisely how much jet fuel we’ll get out of 1 barrel of crude oil. The funnel just confirms what we already expect.

With the ETL pipeline, we aren’t sure beforehand, but, given a chance to inspect the raw data, we could figure out how the end funnel would look like. We can do this because we know the code: we know exactly what the funnel is going to do, and we also have a full view into the raw data.

Things get a bit trickier with the Pachinko machine and conversion funnels.

For conversion funnels, we need to know human psychology. But sadly, Psychology hasn’t reached a state where we can predict people’s behaviour precisely. Thus we have no way to know beforehand how the funnel would look like. We can make guesses, we can look at other people funnels and see how close we are to them, but given none of this extra information, it’s very hard to figure out.

It’s similar with the Pachinko machine. This time, the culprit is Physics. We know the laws of Physics, but we can’t simulate all the forces because we don’t have enough computation capacity.

If you can predict how things would look like in the funnel, the usefulness of actually building and testing the funnel decreases. This distinction is called Gears vs Behaviour. The gears are the inner workings of something, while the behaviour is what you see.

Funnels are great at exposing behaviour where we don’t understand the gears.

Thus, the second core idea of funnels is how much of the gears do you know? When you don’t know the gears, using funnels helps make sense of things. It helps you figure out the behaviour, which is a crucial hint to figuring out the gears.

Funnels as Measurement

Sometimes, figuring out the gears is too hard. You want to stick in behaviour land, but you still want to measure what’s happening, and improve on it.

This is why funnels are so popular in sales and marketing: You’re not trying to push the field of Psychology forward, but using measurement from funnels to drive your product forward.

Once you build a conversion funnel, you see people coming in, you know the conversion rate.

Then, you test changes to your flow, like the colour of buttons, size of the call-to-action, etc. and your funnel conversion rate changes. This is a proxy for human behaviour: you’re figuring out how people react to your product, and using this measurement as a guide for what changes to keep, and which ones to discard.

It’s the same with the Pachinko funnel. Every time you make changes to the board, you do a test with a thousand or so balls, and see how many win the jackpot. (This is your funnel conversion rate). If the number of jackpots gets too high, you know you’re in trouble - you’ll be giving away lots of money.

Conclusion

Funnels are a way to make sense of aggregates: lots of things you don’t understand, each unique in their own way, but put together in a funnel, and they click into place.

They’re lots more actionable than plain goals. A part of this comes from their granularity: while a goal may be “get 100 users to pay”, a funnel would be “get 10,000 users to visit the website, 2,000 to sign up, and 100 to pay”. It gives you more levers to pull, by seeing exactly in which step are people faltering.

Remember though, that these steps need to make sense. Even if granularity is better, you can’t go build a 100 step funnel, since this overfits to the specific experience of a small number of people: you want steps that are important to your goal. And if you have a 100 important steps, you have 0 important steps.

Finally, funnels help measure quality. If you get a shit-tonne more petroleum and not enough jet fuel, it probably means you’re dealing with a lower quality, artificially processed crude oil. We can reach this conclusion because we understand chemistry. However, if we didn’t, it would boil down to (pun intended) testing different kinds of crude oil to decide which one to use - the one which gives us the highest conversion rate.

For another example, consider an expensive marketing experiment. If 100 people paid before the experiment, and a 100 people paid up after, just looking at this number doesn’t tell you much. But consider the funnel:

	Before	After
Visit	10,000	100,000
Sign Up	2,000	4,000
Pay	100	100

Your conversion rate dropped from 1% to 0.01%. That’s a lot more poor quality people entering your funnel. That’s definitely an experiment targeted at the wrong audience.

To summarize: See aggregations, think funnels. See goals, think funnels. See something hard to measure, think funnels.

Thanks to Julian and Nishit for reading drafts of this.

Funnel from sales lingo. The naming is unfortunate, since a funnel is also an instrument in Chemistry. ↩
It’s the value itself! ↩

Building your own Hey email Feed in Gmail

2021-07-17T00:00:00+00:00

If there’s one epic idea Hey.com has, it’s the Newsletter Feed. Like a social feed, you can scroll through these newsletters, read what’s interesting, skip what’s not. It feels natural. It feels unlike regular email.

However, it gets annoying quickly. Some newsletters are very long, the UI is janky, some feed items are randomly skipped, and it’s a pain scrolling through the long emails I’ve lost interest in.

The second epic-in-principle idea is screening people in so random emails don’t reach me. Some companies are pesky and don’t respect unsubscribe buttons. This can get annoying quickly: I have to go the extra step of approving people who I know will email me, or for OTPs coming to my mail.

While I appreciate Hey’s way of thinking about email, their too constrained approach doesn’t suit me. So, I replicated their principles in Gmail, in a way that really works for me.

It took me less than 5 minutes to set up (and more than a few hours to figure out). But, I’ve been using this system for the past 6 months, and recently cancelled my Hey subscription: this approach works so much better for me, and builds into tools I already use.

This is the way: Multiple Inboxes in Gmail.

Here’s how it looks:

This setup works really well for work, too. I have an extra inbox for email from GitHub, which has almost all of my work related emails, neatly separated from the rest, and I can go through them just like this newsletter feed.

The sections below show you how to set it up from scratch.

We’ll work with labels, automatic filters and multiple inboxes. Together, these make Gmail very powerful.

Step 1 -> Enable Multiple Inboxes. Go to Settings -> See All Settings -> Inbox (3rd tab) -> Inbox Type: Multiple Inboxes.

Fill in Section 3 with “is:feed” and then click Save Changes at the bottom.

Step 2 -> Set up automatic filtering for a new label called “feed”.

The video below shows how to do this in depth. You can alse see the official documentation.

Note that we’ll be choosing what to include in the feed, and also what not to. A good idea is to skip the Inbox as well, so these don’t show up as duplicates. If you want to see everything together in one inbox, you can go to “All Mail”.

Here’s how my filter looks:

{from:@substack.com from:@rootsofprogress.org} OR (from:notify@sethgodin.com) OR (“Farnam Street” OR “Idea Muse”)

{} is short form for multiple ORs. Anything without an OR is an AND. I like separating them out to distinguish domains, specific addresses and string searches.

The first part filters in all emails from these domains - {from:@substack.com from:@rootsofprogress.org}. The beauty of this is that all substack newsletters automatically go into the feedbox, instead of manually choosing them one by one.

The second part filters in emails by specific sender - (from:notify@sethgodin.com). I do this when I don’t want mails that aren’t about blog posts in the feed box. For example, notify@sethgodin.com sends blog posts, but seth@sethgodin.com sends other blog updates. I only want one in the feed.

The third part filters in emails by string matching - ("Farnam Street" OR "Idea Muse"). I use this when I’m being lazy, or the newsletter title is catchy, or if I want to see mails that reference these newsletters as well.

You could also filter via just the subject line, like so: OR subject:"Idea Muse". This will only look in the subject, instead of the full email.

Next, whenever you find something to add or remove from your feed, you can edit this filter, like so:

Note:
It’s important to re-edit using the “Advanced Search” button, and not the search bar. Otherwise, you lose the filter.

I do like scrolling through mails together, but I’ve settled on a low-tech, future-proof compromise here.

I setup keyboard shortcuts so that I can quickly switch to the next mail, so if it’s too long I don’t have to scroll to the end.

I use my right hand to scroll through the email, and my left hand is on the 1 and 2 keys. Older mail is 1. Newer mail is 2. Whenever I get bored, I press 1, to see the next feed email. This works surprisingly well.

In normal mode, I keep the feed box collapsed, so I don’t see those mails, and check the feed out in bulk on weekends.

On my real Gmail account, it looks like this:

How to set up screening in Gmail

This is an additional Inbox with a screened-in tag. In the end, the Inbox becomes the graveyard, or an opportunity to filter things in and out.

The filter is what looks different. Follow steps 1-2 above, and then modify the filter like in the video below.

We don’t want to include things that are labelled as feed already, so: “-label:feed” is the first part, and then, I like to add all gmail.com addresses to the filter - this isn’t usually companies, is majorly friends, which means I don’t face headaches having to screen them in all the time.

The full filter looks like this:

{from:@hackernewsletter.com from:@gmail.com from:@neilkakkar.com} -from:hestonb3@gmail.com -label:feed

I add my own email to the not list (hestonb3@gmail.com in this example), so that things I send don’t show up in screened-in.

You’ll note how a “spam” mail I sent from my hey address is still in the Inbox (end of video) - I can now decide whether I want to screen it in, or archive it, or let it be. Screened-in is the main inbox now.

In the end, things look something like:

Notice the timestamps - Inbox fills up much quicker than screened-in, full of mostly crap you don’t want to waste time on.

So, whenever something new comes into the Inbox, and I notice I don’t want it there, I either add it to the screened-in filter or the feed filter.

I generally don’t look at the Inbox, as all important mails are in the screened-in box and the feed is in the feed box.

Note:
Filters are applied in the order they’re written in the Settings, so if your filter includes a label tag, make sure the filter for that label is above this filter!

Another important advantage here is that your Inbox still gets emails you otherwise have not labelled. So, there’s no risk of missing out on important emails from new people.

Oh, and the Hey paper trail is mostly not useful. Not worth setting it up.

I can set all this up once, and then mostly forget about it. It makes things powerful, better integrated, more customizable, and finally, £99 cheaper.

If you know of any other ways to do this that work better, let me know!

Debugging Interesting Bugs at PostHog

2021-05-29T00:00:00+00:00

This week, I ran into an interesting bug. We would send session recording data to our client’s servers instead of our own. This is pretty annoying - we’re losing data, polluting our client’s logs, oh and it happens often, but we can’t reproduce it.

I think it’ll be fun to walk through how I solved this bug. And since PostHog is open source, we can walk through the actual code instead of a make-believe alternative.¹

There’s 4 things I hope you (and I) get out of this post:

Figure out different approaches to solving this bug.
Draw general principles that would be useful elsewhere.
Figure out how to apply these principles more effectively in the future.
Have fun solving this bug

Context

PostHog has a main app: a Django backend and React frontend, which manages analytics. This is where you’d come to see analytics for your website. Then, there’s the integration libraries: you’d run these on your website - a snippet - that manages sending requests to PostHog, auto capturing events, doing session recording, etc. etc.

How PostHog looks like on this website

We’re dealing with the posthog-js library. The session recording extension in PostHog-js manages session recordings. When things work fine, We capture these recordings and send them to the /s/ endpoint on the main app.

When things don’t work fine, we send random requests to /sessionRecording, instead of where PostHog is being hosted.

What makes this bug tricky is that it’s hard to reproduce. We know it happens for a LOT of people, but we don’t know exactly when or how. And this is indeed Principle #1 for any bug you’re trying to solve.

Principle 1: Reproduce the bug

For the easy bugs, a good test is all it takes to reproduce. This makes things so much easier, since if you write the test first, see it fail, and then write the code to fix it, bam, you’re done!

The hard bugs require you to set up the entire flow, imitate the user and see where they run into issues. As you can imagine, the /sessionRecording bug is of the second kind, which is what makes it blogpost worthy.

PostHog: Reproducing the bug

Well, before I can reproduce anything, it’s a good idea to setup my development environment, so everything is local, and I can tweak things the way I want to. This bit is interesting in itself, since setting things up is non-trivial: you create a sample website to use as a playground, add the snippet that talks to PostHog, and run PostHog locally to ingest those events. There’s a bit more to it, but it’s probably not worth getting into it right now.

Since we’re dealing with a local web server, I set up http-server - which would tell me all about the requests coming in. Now, I somehow need to make the /sessionRecording request come in. Enable session recording on the server side, enable it on the client side, some fumbling around, refresh, and sure enough, I got a few /sessionRecording requests.

I realised I’d get a new /sessionRecording request 50% of the time after a refresh. I considered this a good enough rate to dive deeper.

Next step is to figure out which one of the 30,000 lines of code is the problem. I’ve found that the best way to figure anything out is to make hypotheses, test if they’re right or wrong, and repeat. Every wrong answer gives you more insight into what to try next. If this sounds familiar, yes, it’s exactly the same as the framework for critical thinking.

PostHog: Testing what went wrong

The first thing I’d do is grep for sessionRecording. Find where the damn URL is being generated, and we’re done! In fact, I did this even before trying to reproduce the bug - sometimes, that’s all it takes.

Well, this bug was blogpost worthy, so it obviously wasn’t that simple. You can search for sessionRecording in the repo . The only place worth looking is in sessionrecording.js, which doesn’t make sense. Maybe the string is being generated somehow? or a third party library is doing it?

Note:
If you’re following along, use this non-master branch, since I’ve already merged the fix into master

My focus areas now would be every part of posthog-js that makes an HTTP request somewhere. If requests are being made from the client, the DevTools must have captured this request too!

That’s where things start to get even weirder. I could see the requests coming to my http-server:

but not on the DevTools network tab:

Can you guess why?

The /sessionRecording request would come right after I hit refresh.² …Maybe I was wrong about this, and it was right before refresh. Since I wasn’t preserving the logs, those requests would vanish.

Sure enough, preserving the logs showed the request:

The first thing that caught my eye was how this was different from a regular XHR request. A ping ? What, how is this happening? A few google searches led me to the ping attribute of the anchor tag. Is there code turning HTTP requests to <a> tags? Not sure. I don’t see the DOM changing. This track didn’t lead me anywhere. Maybe other things can generate a ping type request, too?³

So, I decided to move on. Here’s what we know so far:

These requests are made by the client.
They’re made right before refreshing.
They’re different from usual XHR requests.

It smells of a cleanup script running, but the codebase is huge: hard to traceback where this is coming from yet. So, it’s time to get dirty with the code to figure out the traceback.

Well.. let’s console.log this shit up. (Yeah, I know now this is ~~a bit~~ very n00by). I’m looking for the url parameter, specifically where it equals sessionRecording. It took a while, working my way up to the right piece of code.

We’ll skip all that n00b logging and move on to the final piece. The culprit is somewhere in here:

Full file on Github

Can you guess what’s wrong?

I’ll help with some console.logs: this.formatQueue() returns something like:

Rebuilding that myself:

{ 'sessionRecording': { '/s/', binary_data, {transport: 'sendbeacon'}},
  '/e/': {'/e/', binary_data, {transport: 'sendbeacon'}},
}

The key is sessionRecording because of formatQueue() - the batchKey takes precedence over the url.

This should make things clear. The bug was because of unload(). Here’s the final PR.

I think there’s lots to generalise from this experience, so let’s ground out the remaining principles.

Principle 2: Make and test hypotheses

What distinguishes fast debuggers is how quickly they cut down the search space by making good hypotheses.

The 30,000 lines of code are your search space. Through testing hypotheses, you discard lines of code that can’t possibly be the problem.

Grepping is useful because if it works, it narrows the search space down to a few lines of code.

Making and testing hypotheses is a bit trickier than it sounds. For example, consider when the /sessionRecording requests happen. I’d refresh my local website, switch screens to the server logs, and sure enough, I’d find the /sessionRecording POST request there (sometimes). This information wasn’t enough to figure out whether the request happened right before, or right after the request.

Computers are much faster than humans, so our usual intuition about timings don’t work here. I inferred the test results incorrectly. I should’ve seeked disconfirming evidence, like every hypothesis demands.

PostHog: Doing things better

Coming back to our bug, I’m still not happy. I was pretty slow. This was n00b level debugging - as you can see from the 100s of console.logs I put in the code. There has to be a better way.

Making and testing hypotheses is a 2 part problem. The first part is making hypotheses, which depends on your brains and how well your tools uncover places to look.

The second part is testing those hypotheses. I was pretty slow here, because I wasn’t relying on the right tools. I figured two optimisations I could make:

Switching my development settings so instead of a minified js code for the client, I either get a source map, or non-minified code.
Using breakpoints in Chrome DevTools (it’s surprising how amazing these have gotten)

I didn’t realise until after that both of these were available options. So, I tried debugging the code again, using chrome dev tools.

Chrome DevTools

This is pretty cool: it shows me what console.log would have, but better: I don’t have to log everything myself.

zoomed in

… However, it didn’t help me debug this specific problem better: unload events are unreliable, and Chrome does a poor job of breaking on unload. Stack Overflow had some answers but none worked as well as I’d expect. Disappointing, but I imagine this would work well for something happening not at the end of a page load.

Principle 3: Better tools make things easier

Imagine people in a jungle shouting “it’s here!” to indentify target animals, instead of using a radar. It’s harder, unreliable, and doesn’t always give the full picture. That’s what console.logging is, compared to a debugger.

But hey, sometimes, radars don’t work either, and you have to revert to more primitive options, like in the example above. If there’s one thing, it’s that console.log is resilient.

I feel on aggregate, I’d end up doing things faster with a ~~radar~~ debugger: it’s the difference between seeing only what I focus on - since I’d log things I want to focus on - versus seeing everything, and choosing what’s interesting and what’s not.

Principle 4: Write Tests that fail

So, we’ve figured out the bug. Do we just fix what was wrong and call it a day? Nope, never!

Once you think you’ve figured things out, you need to test your solution works, just like your hypotheses! And since we’re seeking disconfirming evidence, it’s crucial that your test fails before the fix, and works after the fix.

This doesn’t mean TDD, or writing your test before your fix, but it does mean running your tests before and after the fix. If you noticed the bug fix PR above, you’ll see how I wrote the test afterwards. What the PR doesn’t show is my workflow that gave me confidence in the fix and the test.

My workflow goes something like: I write the fix, commit to a git branch, then I write a test and see it pass. Now, before committing the test, I git checkout HEAD~3 or git checkout master (i.e. go back 3 commits: or however many it took to fix the bug), then run the test, and see it fail.

If the test on master passes, either you’re testing the wrong thing, your test doesn’t work, or your fix didn’t actually fix the problem, since the test passed both before and after.

It’s this “watching things fail –> writing code to fix –> watching things pass” loop that grants confidence in your code.

End Notes

One mystery remains: where does the ping request type come from? The hint was in {transport: 'sendbeacon'} above. This link is very interesting!

And, that’s all for the session recording mysteries! Caught your fancy? PostHog is hiring.

What did you think of this post? Let me know on Twitter. I’m super excited about all the experiments I can run, working with an open source company!

I’m probably more excited about the “actual code” bit than the “bug” bit. I’m looking forward to all the experiments I can run working at an open company. This is #2. ↩
This was me incorrectly inferring the test data. Since I’ll click refresh, then switch to the server logs, it made it seem like it happened after refresh, when in fact, it was right before. ↩
Yes, they can! See the end notes for more information. ↩

Why Is Naming Things Hard?

2021-02-16T00:00:00+00:00

An interesting shift happens once you realise you’re writing code for humans to read, and not just for machines to execute.

One big change is that writing clearly takes priority over correct code.¹

Reading code involves building up a mental model of what each thing should do, and how these things interact with each other. The best code, however, short-circuits reading everything. Instead of reading the implementation, you use variable and function names, comments, and other clues to figure things out.²

This is chunking. You combine things together, and remember them together. For example, a Chess Opening, like the Queen’s Gambit, is a chunk. So is a forEach loop: just by the name, you can figure out that it’s going to do something to every element of an array.

Once you’ve built a chunk for a forEach loop, you don’t have to read the implementation of forEach to understand what it will do. This is very powerful, since it allows you to come back to the code, without having to read the implementation of everything again.

Thus, a good name is a chunk that encodes important information about what the thing does.

We’re compressing information into the name. This is almost always a lossy compression. Further, the more lines to compress, the more losses you incur, since you want your names to be no longer than 80 characters.

The goal of good naming is to minimize this loss.

And naming things is hard because of this compression. English isn’t precise like C++, so compressing from a precise to an ambiguous language increases losses. Another big reason is that choosing the most important things to include is hard. You’ll notice that most disagreements about naming fall on these two axes: either the name isn’t just right for what you want to do (does it assign or setup or link or mangle?). Or the name doesn’t capture important parts of the functionality.

I think this is the generating function for most best practices I’ve heard of, and thus lots more valuable than one specific guidance.

Generating best practices

Make functions do one thing

When a function does multiple things, the name needs to encode lots more information, like link_user_to_trial_and_enable_downloads_and_set_charge_date(). That’s a long name, and a sign of losing too much information: what semantics should the name expose about the charge date, and enabling downloads?

These side effects are hard to encode for in the name.
Keep functions small

Long functions, like functions doing lots of things, need to encode lots of information into the name. And since the name length is bounded, you lose a lot more information in compressing this long name, which leads to poorer names.
Avoid meaningless names

Since we’re writing code to be read by humans, names that don’t compress any information don’t help us build chunks. This makes it harder to read, since you have to go down one layer of abstraction. You need to parse the implementation to understand what that piece of code does.
Avoid too abstract names

do_things() is a suitable name for any function, but the compression is too lossy to be useful - you lose almost all information, and need to parse the body to figure out what it does. Not great. It’s almost a meaningless name.
Be consistent

When you build similar chunks throughout your code base, it becomes easier to go up another layer of abstraction, by chunking the chunks. For example, continuing the trial management system example, you’ll have 3 things in place:
1. link_user_to_trial_agreement_given_email()
2. enable_research_downloads_for_user()
3. set_charge_date_for_user()
Since they’re operating in the same domain, you can go up one level to setup_user_for_trial() that does all three.

Disrupting best practices

The generator of best practices allows you to reason about cases where best practices aren’t the best way forward.

For example, consider being consistent, and naming variables using camelCase. Your entire codebase uses camelCase. You’re writing a new function that’s very non-standard. It does arcane stuff that doesn’t fit into your existing model. You need it, because legacy integration/business requirements/etc. In effect, you don’t want anyone to gloss over it when they read it.

So, do you stay consistent, and keep it in camelCase? Or, do you switch to snake_case?

The goal is to keep things clear. I would switch to snake_case to show how it’s different from the rest, and have some comments explaining the arcane things it does.³

Precision with words

Spoken languages are very imprecise. There’s lots of ambiguity, double meanings, and sentences that don’t mean anything.

Code, however, needs to be precise. Like we saw above, this is a source of trouble, since we’re trying to compress a precise language into an imprecise one, which makes searching for the right English words harder.

These ideas apply to naming things outside of code, too! When you have a name for something, it suddenly becomes legible.

Consider: You see some people around you who test if they can do something once, and drop it if they fail. Some others keep trying until they learn how to do it. The first group prizes its intelligence, while the second group prizes its hard work. Then, when you hear about the Fixed vs Growth mindset, things immediately click.

Choosing the right name here is hard too. Every best practice that lost its nuance over time can be attributed to a name that didn’t encode the necessary information. People passed these ideas on without explaining the nuances.

Or, consider equating two names. “Abortion is murder” - equating two names that aren’t the same creates an interesting dynamic between emotions and the truth. Each name has different connotations, and on combining them together, you’re passing on the connotations of one onto the other. This isn’t necessarily a good thing. You have to be very careful of the equality operator in an imprecise language.

Conclusion

Optimise for clear, easy to understand code. The cost of reading it is higher than the cost of executing it.

To do this, leverage chunking: write names that are meaningful, and encode as much information as possible into the name.

However, writing meaningful names is a lossy compression. Every naming maxim follows from minimising this loss.

And finally, the same idea applies to naming things in the real world. Except, it’s even harder, since it’s encoding from an imprecise to another imprecise language. Code, atleast, was unambiguous in its execution.

Thanks to Soumya for reading drafts of this.

Have feedback? You can reply to this post on Twitter.

You might contest this, that correctness comes first, but if it’s clear to read, PR reviews will catch correctness bugs quickly. Or, if no-one catches mistakes, and then things break, it’s easier for your colleagues to fix things if they can read your code clearly.
Footnote to footnote: This is an instance of taking ideas seriously. ↩
Sometimes, poor naming gets in the way, generating an incorrect model of how things work. Here, you have no choice but to read the implementation. ↩
Ideally, you’d fix things properly so there isn’t a need for something like this. But, the default is to have things like these littered over a huge codebase. ↩

Neil Kakkar

Agentic Debt

The feedback loop

The context window trap

Human-understandable is agent-actionable

Tending the garden

Open questions

What I learned about burnout and anxiety at 30

The slow slide

The signs I missed

Rebuilding

What I’ve learned

Environment > willpower

Stress reveals itself through control

The same patterns repeat

I sacrifice stability for growth

How to setup duration based profiling in Sentry

Appendix: Sample transaction

How to simulate a broken database connection for testing in Django

Option 1: Patch raw cursors

Option 2: Patch Django query internals

Greatest Option 3: Leverage database wrappers

Why is this useful?

The “People fuck up because they’re not like me” Fallacy

How I Own Projects as a Software Engineer

Gather Context

Figure out a solution

Build

Gather Feedback

Align metrics with feedback

Meta: Create feedback loops

Meta: Manage your Emotions

Conclusion

Funnels: The One Big Mental Model from Sales & Marketing

Crude Oil Distillation

ETL Pipeline

Conversion Funnels

Pachinko

Where Funnels make sense

Funnels and Gears

Funnels as Measurement

Conclusion

Building your own Hey email Feed in Gmail

How to create a Newsletter Feed in Gmail

How to scroll through a newsletter feed in Gmail

How to set up screening in Gmail

Debugging Interesting Bugs at PostHog

Context

Principle 1: Reproduce the bug

PostHog: Reproducing the bug

PostHog: Testing what went wrong

Principle 2: Make and test hypotheses

PostHog: Doing things better

Principle 3: Better tools make things easier

Principle 4: Write Tests that fail

End Notes

Why Is Naming Things Hard?

Generating best practices

Disrupting best practices

Precision with words

Conclusion