Posts on Manthan

What Happens When You Ask ChatGPT a Question

Thu, 17 Sep 2026 00:00:00 +0000

You type a question into ChatGPT and hit send.

A JSON body leaves your laptop as an ordinary HTTPS POST. The same POST, more or less, whether the box you typed into was ChatGPT, Claude, Cursor, or a curl against OpenAI’s API. A few hundred milliseconds later, a token shows up, then another, then another. Most of us treat that gap as a black box: text went in, tokens came out. Fine, until you care why the first token paused, why a short answer cost a long bill, or why the same prompt is cheap on the second call and expensive on the first.

How to Evaluate Voice Agents

Sat, 15 Aug 2026 00:00:00 +0000

A voice agent can pass every eval you wrote for it and still be broken in production. The transcript reads correctly, the tool call fires with the right arguments, the backend state lands where it should, and the scorecard marks the call a success. Then you listen to the actual recording and hear the agent talking over the caller, mangling their name, and leaving three seconds of dead air before it confirms anything.

How Does Cursor Index Your Codebase?

Tue, 16 Jun 2026 00:00:00 +0000

Every time you ask Cursor “where do we handle authentication?” and it points you to the right file in a 50,000 file monorepo in under a second, something interesting has happened under the hood. It’s not magic but an elegant combination of Merkle trees, trigram indexes, AST-based chunking, a custom trained embedding model, and a vector database (turbopuffer) storing over a trillion vectors across 80 million namespaces. This blog post is special because I am a big fan of Cursor and Turbopuffer. So I thought it would be interesting to understand how it works under the hood. Let’s get into it.

Memory in Voice Agents Is a Harder Problem Than You Think

Tue, 12 May 2026 00:00:00 +0000

If you wanted the voice agent pipeline — STT, LLM, TTS, the latency budget — that is the architecture post. This one is the sequel that actually breaks the budget: memory.

The first time I tried to plug a memory layer into a voice agent, the thing slowed down by hundreds of milliseconds on the first turn and never recovered. The conversation went from natural to “is the line still there?” in a single round-trip, because I had ported a text-agent memory architecture wholesale: a synchronous vector lookup against a hosted store, a small re-ranking pass, then the LLM. On a chat surface, nobody would have noticed. On voice, every turn felt like the agent was buffering before it spoke.

Voice Agent Architecture: Cascade, Latency, and the Pipeline

Tue, 05 May 2026 00:00:00 +0000

The first time I built a voice agent and got on a call with it, the thing felt like talking to someone over a satellite phone in 2003. Long pauses, unnatural cadence, occasional cuts where it would just keep talking over me. The text version of the same product was usable. The voice version was not. That was the moment I learned that text engineering and voice engineering are not the same craft.

Reverse Engineering ChatGPT, Claude, OpenClaw, and Hermes Convinced Me Most AI Products Shouldn't Ship Memory

Wed, 22 Apr 2026 00:00:00 +0000

The first time I asked ChatGPT what it remembered about me, it listed 33 facts. Name, career goals, fitness routine, names of side projects I had mentioned weeks earlier, a throwaway line about my sabbatical from a completely unrelated chat. I was impressed. I spent the next few weeks reverse-engineering how it actually works, then did the same for Claude, OpenClaw, and Hermes. Somewhere around the third system, I stopped noticing how clever the designs were and started noticing how often memory was degrading my own outputs.

What Happened When I Applied Karpathy's Autoresearch Idea to LLM Inference

Sun, 22 Mar 2026 00:00:00 +0000

Most “AI optimization” demos are fun to watch for the same reason benchmark tweets are fun to watch: they show you the win, not the search.

You see the final graph. You see the +12% or the “runs 2x faster now” claim. What you usually do not see is the graveyard of bad ideas behind it. The settings that looked promising but were just noise. The optimizations that made throughput better by quietly making the model worse. The fake wins that only happened because the benchmark got easier.

I Read Hermes Agent's Memory System, and It Fixes What OpenClaw Got Wrong

Mon, 16 Mar 2026 00:00:00 +0000

If you’ve read my previous posts on ChatGPT memory, Claude memory, and Clawdbot memory, you already know I keep coming back to the same question: how do these agents actually remember?

Hermes Agent was particularly interesting to me because this time I did not have to reverse engineer everything from behavior alone. Hermes is open source, and both the repo and the docs are public. So instead of poking a black box with prompts, I went straight to the code paths that build prompt state, persist sessions, flush memories, and query past conversations.

Karpathy Autoresearch: How the Experiment Harness Works

Fri, 13 Mar 2026 00:00:00 +0000

Most “autonomous AI research” demos look impressive for the same reason magic tricks do: you only see the interesting part. An agent edits some code, runs an experiment, and shows a better result. What you usually do not see is the part that actually determines whether the system is useful: what is the harness optimizing for, how stable is the evaluation, and what happens when the agent fails?

That is why Karpathy’s Autoresearch is worth paying attention to.

How OpenClaw Remembers Everything

Mon, 26 Jan 2026 00:00:00 +0000

OpenClaw is an open-source personal AI assistant (MIT licensed) created by Peter Steinberger that has quickly gained traction with over 32,600 stars on GitHub at the time of writing this blog. Unlike ChatGPT or Claude which run in the cloud, OpenClaw runs locally on your machine and integrates with chat platforms you already use like Discord, WhatsApp, Telegram, and more.

What sets OpenClaw apart is its ability to handle real-world tasks autonomously: managing emails, scheduling calendar events, handling flight check-ins, and running background jobs on a schedule. But what caught my attention was its persistent memory system which maintains 24/7 context retention, remembering conversations and building upon previous interactions indefinitely.

Agentic System Patterns That Increased Accuracy by 50% (And What They Cost)

Tue, 20 Jan 2026 00:00:00 +0000

Agentic systems are already being deployed for complex tasks: building software, conducting research, analyzing data, and automating workflows. But as they move from prototypes to production, teams face a fundamental question: how do you build an agentic system that reliably handles any task?

The answer isn’t just about better prompts or more sophisticated models. It’s about understanding the trade-offs between three critical dimensions:

Cost - API calls, compute resources, infrastructure
Latency - Time to complete a task, user experience
Accuracy - Correctness, reliability, handling edge cases

The truth is that most techniques that improve accuracy also increase cost and latency. Multi-step reasoning, parallel verification, self-correction loops. They all make your system better, but at a price. The key is knowing when to pay that price and how to optimize within your constraints.

How LLM Tokenization Works, and How to Train Your Own Tokenizer

Wed, 14 Jan 2026 00:00:00 +0000

TL;DR: Modern LLMs use subword tokenization (BPE, WordPiece, or Unigram) to balance vocabulary size with sequence length. Tokenization directly affects API costs, training compute, and model capabilities. It’s why LLMs struggle with arithmetic and spelling. For custom domains, train your own tokenizer using HuggingFace’s tokenizers library. Use Tiktoken for speed, SentencePiece for multilingual.

The wrong tokenization strategy could be costing you $500K+ annually.

How to Use LLM as a Judge (Without Getting Burned)

Wed, 31 Dec 2025 00:00:00 +0000

TL;DR: LLM as a judge works well when you (1) use reference based evaluation over pointwise scoring, (2) debias pairwise comparisons by swapping positions, (3) ensemble across multiple models, (4) require reasoning before scores, and (5) calibrate against human judgments. Skip it when ground truth exists or for safety critical decisions.

Last month, I ran thousands of evals on an agentic system I was building. The responses were open-ended, context dependent, and impossible to validate with simple string matching. I tried exact match comparisons, they failed miserably. I tried BLEU scores, they penalized semantically correct but differently worded responses. I was stuck! Traditional metrics could not handle the nuance I needed, and manual evaluation was too slow and expensive.

I Reverse Engineered Claude's Memory System, and Here's What I Found!

Sun, 14 Dec 2025 00:00:00 +0000

When I reverse-engineered ChatGPT’s memory system, I found it uses pre-computed summaries injected into every prompt. But Claude’s approach is different. Through extensive experimentation, I discovered Claude uses on-demand tools and selective retrieval, a fundamentally different architecture. But how does this actually work? And how does it compare to ChatGPT’s approach?

This is the second post in a series where I reverse-engineer the memory systems of popular AI assistants. The first post focused on ChatGPT’s memory system. In this post, we’ll do the same exercise for Claude—and the differences are fascinating.

I Reverse Engineered ChatGPT's Memory System, and Here's What I Found!

Tue, 09 Dec 2025 00:00:00 +0000

When I asked ChatGPT what it remembered about me, it listed 33 facts from my name and career goals to my current fitness routine. But how does it actually store and retrieve this information? And why does it feel so seamless?

After extensive experimentation, I discovered that ChatGPT’s memory system is far simpler than I expected. No vector databases. No RAG over conversation history. Instead, it uses four distinct layers: session metadata that adapts to your environment, explicit facts stored long-term, lightweight summaries of recent chats, and a sliding window of your current conversation.

How I Got Every Job Without an Interview

Sun, 23 Nov 2025 00:00:00 +0000

I have never relied on traditional interview processes to get a job. No DSA rounds, no system design grilling, no multi-step loops. Almost every role I have landed came through a single conversation focused entirely on the projects I had built.

The reason is simple: proof of work makes interviews optional.

I didn’t come from a big-name college. There was no built-in reputation attached to where I studied, and no advantage from well known tags like IIT, GSOC, or Big Tech internships. Those signals help - they make it easier for someone to assume you are capable.

Towards Human-Like Memory for AI Agents

Sat, 01 Nov 2025 00:00:00 +0000

Memory has become one of the most talked about topics in the AI community. Every few weeks, we see new papers, frameworks, and posts, each claiming to “solve” memory for LLMs. Yet most of them are glorified caches that store tokens, not thoughts. Today’s memory systems rely on brute force retrieval: dumping everything into a vector database or rank fusing it with BM25. These approaches scale clutter, not cognition.

If we want agents that truly understand and evolve with users, we need something deeper: metacognition. In this blog post, I will share how I believe the memory layer of agents will evolve to become more human-like.

Prompt Engineering: Steering What the Model Predicts Next

Mon, 11 Aug 2025 00:00:00 +0000

LLMs are prediction engines. The model takes text as input and then predicts the probability distribution of the next token based on the data it has been trained on. The LLM repeats this process multiple times, adding the previously predicted token to the input and predicting the next token. When we prompt the model, we are steering the model to predict the right sequence of tokens. Prompt engineering is the art of crafting the right prompt to get the model to predict the accurate output.

Water: An Agent Harness, Not Another Agent Framework

Sat, 07 Jun 2025 00:00:00 +0000

Building multi-agent systems with existing frameworks is painful. While OpenAI Swarm, Google ADK, LangChain, CrewAI, AutoGen, Agno, etc have orchestration capabilities, they are complex, verbose, and unintuitive. Simple workflows require dozens of lines of boilerplate code, and adding branching logic, retries, or conditional execution becomes a debugging nightmare.

Water is an agent harness framework — it provides the infrastructure around your AI agents, not the agents themselves. Orchestration, resilience, observability, approval gates, sandboxing, and deployment tooling so you can focus on what your agents actually do.

Taking Sabbatical at the Age of 24

Wed, 01 Jan 2025 00:00:00 +0000

This is a non-technical post that I feel fit to share as the year 2024 has come to an end. It’s been a little more than a year since I took a sabbatical and joined the workforce in 2024. I will cover everything from why I took a sabbatical to what I did during the sabbatical and how I spent my time.

Why I Took a Sabbatical?

Taking a sabbatical at the age of 24 isn’t a typical decision, but in September 2023, I chose to step away from work for some much-needed time off. This decision wasn’t easy, but it was necessary for several reasons:

Write Skew, Dirty Reads, and Weak Isolation in Distributed Transactions

Sun, 01 Dec 2024 00:00:00 +0000

Distributed databases usually will not give you serializability. They give you weak isolation, which is a polite way of saying some concurrency bugs are still allowed through: dirty reads, dirty writes, lost updates, write skew, and phantoms.

This post is how those anomalies actually show up, and which isolation level stops which one. What ACID means, and when a single-object write is not a transaction is the setup. Let’s get into the isolation levels.

What ACID Means in Distributed Transactions

Fri, 01 Nov 2024 00:00:00 +0000

A transaction is a way to group several reads and writes so they succeed or fail together. That sentence is easy. Making it true across nodes is the rest of this series.

This part is the vocabulary: what ACID actually means, and why a compare-and-set on one row is not the same thing as a multi-object transaction. Next is write skew, dirty reads, and weak isolation.

What are transactions?

A transaction is a logical unit that groups several reads and writes. All the reads and writes in a transaction are executed as one operation. Either the entire transaction succeeds or fails. If it fails, it is safe to retry, as all the changes have been reverted inside the transaction.

Partitioning in Distributed Systems: Hot Spots and Rebalancing

Tue, 01 Oct 2024 00:00:00 +0000

Partitioning splits a database across nodes so one machine is not holding all the data or all the load. The hard parts are skew, hot spots, and moving rows when the mapping changes.

This post is those problems: how you partition, how you relieve hot spots, and how you rebalance. Replication is the other half, starting with why databases replicate.

Partition & Replication

We have already discussed replication in depth in my 3 part series. If you have missed it, you can find the 1st part here.

CricLang: A Programming Language for Cricket Fans

Fri, 01 Mar 2024 00:00:00 +0000

CricLang is a small programming language for cricket fans, built in Go. The first commit on the repo is June 9, 2023, but the idea had been sitting around since college. I actually started building it on Feb 17, 2024, and it is now in public beta.

Why the name CricLang?

CricLang is an amalgamation of ‘Cricket’ and ‘GoLang’ as it is built on top of Go. It represents my love for cricket, software engineering, and the idea that code should be as simple and readable as the game. It is an evolving programming language and isn’t perfect by any means.

Leaderless Replication: Quorums, LWW, and Version Vectors

Sun, 11 Feb 2024 00:00:00 +0000

Leaderless replication has no failover because there is no leader. The client writes to several replicas, waits for a quorum, and lives with the mess when nodes disagree: read repair, anti-entropy, last-write-wins, and version vectors.

If you want the leader-based half of this series first: why databases replicate and read-your-writes, monotonic reads, and multi-leader replication.

Leaderless Replication

In a leaderless implementation, the client directly sends the writes to several replicas and waits for the majority of them to acknowledge, compared to a setup with one or more leaders where a leader exists to coordinate the writes.

Read-Your-Writes, Monotonic Reads, and Multi-Leader Replication

Thu, 01 Feb 2024 00:00:00 +0000

Asynchronous replication makes a database fast and also slightly dishonest. You can write a comment and immediately fail to see it. You can refresh and watch time go backwards. You can read a reply before the message it is answering.

Those are consistency guarantees weaker than “the database is correct,” and they have names: read-your-writes, monotonic reads, and consistent prefix reads. This post is those guarantees, then multi-leader replication and how write conflicts get resolved. The setup is leaders, followers, and failover.

Why Databases Replicate: Leaders, Followers, and Failover

Fri, 26 Jan 2024 00:00:00 +0000

You replicate a database so one machine is not the entire story: more read capacity, a spare if a node dies, and a copy closer to the user. The usual shape is a leader that takes writes and followers that replay them. This post is that shape, plus what happens when the leader fails.

It is part 1 of 3 on replication in distributed systems. Next is read-your-writes, monotonic reads, and multi-leader replication, then leaderless replication, quorums, and version vectors.

Data Structures That Power Your Databases

Sat, 20 Jan 2024 00:00:00 +0000

Have you ever thought about why databases are so complex internally and why we can’t use a text file to store the data? This blog post aims to go from the most basic database that writes data to text files to a more complex setting where we use LSM-tree and B-tree. We will understand why we need these data structures and more.

World’s Simplest Key-Value Store

What does the world’s simplest key-value store look like? 2 bash function that writes data to a text file and gets data from the text file.