I love this detailed write-up of the Twitter backend as it exists today. Sending a tweet to your followers uses a massive Redis cluster with a couple terabytes of RAM:
“Let’s say you tweet and you have 20K followers. What the fanout daemon will do is look up the location of all 20K users inside the Redis cluster. Then it will start inserting the Tweet ID of the tweet into all those lists throughout the Redis cluster. So for every write of a tweet as many as 20K inserts are occurring across the Redis cluster.”
A listener of Core Intuition asked me today if I had second thoughts about no longer using Twitter, especially since I still maintain a Twitter app. We’ll discuss this on a future episode, but the short answer is: no, I don’t regret it. I have a huge amount of respect for what Twitter has built at a technical level, and for the opportunities it gives people to communicate. I just don’t like their attitude toward developers.
