An LLM can generate a CNN, a transformer, even a novel architecture. What it cannot do is tell you whether you should have built one in the first place, whether your evaluation is honest, why your model silently degraded in production last Tuesday, or whether the metric you’re optimizing actually corresponds to the business outcome you want. Every one of those questions requires the fundamentals.

What fundamentals actually buy you (and tools never will)

Problem framing. The hardest decision in any ML project isn’t the architecture but it is whether ML is the right tool, what the loss function should encode, what data to collect, what to optimize. People without fundamentals often overcomplicate things and make the wrong decisions. For example, they would reach for ML when a heuristic(a simple rule based system) would do, or for an LLM when classical ML would be 100x cheaper and more accurate. An LLM will cheerfully build whatever you ask for; it won’t tell you that you asked for the wrong thing. For example, AI won’t push back if you asked it to build a candidate filtering system based on resumes you put into it. It will build it for you without telling you that a simple rule based algorithm of your key filters like “work experience”, “education”, “core skills” would work just as well and would be a lot cheaper for you.

Evaluation literacy. This is the single biggest separator of good ML practitioners. Knowing why accuracy lies on imbalanced data, when offline metrics diverge from production behavior, how to construct a holdout that isn’t contaminated, what AUC actually measures or when a benchmark is gameable(meaning on paper you get the job done but you haven’t been able to solve the real problem). Without this, you ship models that “test well”, of course, but fail silently in production.

Failure mode reasoning. ML systems fail in subtle, non-obvious ways: label leakage, distribution shift, drift, calibration breakdown, prompt injection, retrieval poisoning. If you take away one line from this whole write up on fundamentals, let it be this- you cannot debug what you don’t understand. The LLM will suggest five fixes; only fundamentals tell you which one applies to which scenario.

Cost and architecture trade-offs. Knowing when a 50ms logistic regression on a CPU beats a billion-parameter LLM. When fine-tuning beats prompting. When RAG beats fine-tuning. When a distilled 7B model is the right answer over Claude or GPT. This is the difference between burning cash on the wrong approach and shipping a sustainable product.

And if I can say it louder for everyone at the back-LLMs ARE ML. Tokenization, embeddings, attention, context windows, sampling temperature, RAG, fine-tuning, evaluation harnesses and I can go on- these are all deep learning fundamentals. The people getting the most out of LLMs and building real technical products are the ones who understand them to the very core. Everyone else hits the ceiling quite fast.

Tools have a half-life of 18 months. Fundamentals have a half-life of decades. Linear algebra, probability, optimization, generalization, evaluation - these will be just as relevant in 2040 as today. LangChain probably won’t be relevant in 2027. Just as we all wanted to learn Hadoop 10 years ago until better things like DataBricks came along, tools lose their value as a skill sooner rather than later. What remains still is the fundamentals of Distributed computing, data partitioning and fault tolerance. You can move on to the “better” tools built on the same architecture because you know how those bricks hold the structure together.

As more systems become data-driven and probabilistic, the core engineering skill of the next 20 years is judgment under uncertainty about systems that fail silently. That skill is ML/DL fundamentals. It’s not optional anymore as one would think it is, in fact it’s becoming the baseline for what it means to be a technical engineer at all.

As we see, the utility and the spread of generative AI platforms & tools are increasing day by day, and the definition of the “education on AI” is being correlated with the knowledge of AI tools.

Can someone really be called an AI or ML engineer if they know how to use AI tools?

The tool-first philosophy narrative: since we have democratized code generation and absolutely anyone can now write code turning it into a commodity- traditional computer science fundamentals, AI algorithms, and machine learning fundamentals do not have as much of an important role to play in.

However, if we look at the philosophies held by all the top tech industry leaders like Demis Hassabis, Andrej Karpathy, Ilya Sutskever, Yann LeCun and Jeff Dean, it reveals that the opposite is true.

In this era of high-velocity tool-flooded ecosystems and AI-assisted coding development, deep foundational knowledge is not a legacy requirement but a primary differentiator and a competitive advantage of an elite engineer.

History

History has always been our guide, especially when we want to build an opinion on the shifts that we have seen in engineering and technology. To understand this current rise of AI as a technology, one must put in context the broader history of engineering, technology, and computer science. The progression of programming and programming-based tools has always been towards a higher layer of abstraction- meaning- hiding the “how and whys” of the layers beneath it to let you just focus on the outcome it can produce. Every few years, we grow one layer above in abstracting the programming or in abstracting the coding layer. This has led to massive development in the technology space but I am here to tell you not to discount the “how and whys” so soon. Let’s look at some pivotal moments in computing and engineering history:

1950’s

In the mid-20th century, programming was an exercise in “hand-to-hand combat” with hardware. The first generation of software development relied on machine code- direct manipulation of binary (1s and 0s) or hexadecimal representations of data. At this level, there was no distinction between the “tool” and the “fundamental.” Early programmers worked directly with machine code, manually specifying opcodes and memory addresses, and even computing jump targets by hand.

Because this process was highly complex and only understandable by experts, programming was often described as a kind of “priesthood” and only a small cohort of programmers existed who had access to this kind of technology and the intellect to control it.

The second generation introduced assembly language, which used symbolic mnemonics (opcodes like MOV, ADD, XCHG) to represent machine instructions. While this made code more readable, it remained “machine-dependent” and required a deep understanding of processor architecture, registers, and memory alignment.

Compilers

The 1950s saw the birth of the first high-level language, Fortran(Formula Translation), which allowed engineers to write logic in “natural mathematical notation”. Engineers could write expressions like: x = a + b * c and the system would execute it. Hence we created a higher layer of abstraction.

Naturally, as human behaviour goes, the introduction of “compilers” - tools that automatically translated high-level logic into machine code was met with skepticism from the established “priesthood of programmers.” The arguments used in 1958 & 1959 are strikingly similar to those used by modern critics of AI-assisted coding:

1. Efficiency: People from the programming community argued that the compiled code could never be as compact or as efficient as handwritten assembly. They also believed that a human brain would always beat the compiler’s optimization.

2. Control: They feared that surrendering control to an automated translator would lead to bugs they couldn’t debug and systems they didn’t fully understand. (It sounds a bit familiar?)

3. Trust: If the engineer did not “code to the metal,” how could they be sure the machine was executing the intent correctly?

Despite this resistance, by 1958, more than half of the code running on IBM computers was generated by the Fortran compiler. Crucially, this did not succeed just because the tool existed now, it worked because engineers in time started to trust it while having a solid understanding of the layer beneath it.

What do we really learn from going through all of this history?

We can derive that the most elite successful engineers were not those who refused the “tool” (the compiler) but those who mastered the high-level logic while maintaining enough foundational knowledge to understand how the compiler optimized their intent. This transition did not destroy the need for fundamentals; it shifted the fundamental focus from low level concerns like “memory address management” to higher level thinking like “algorithmic logic and problem decomposition”.

Progression of Abstractions

1940s–50s: Machine Code

1. Abstraction Level: Machine Code

2. Primary Interaction: Binary / Hex Strings

3. Fundamental Requirement: Hardware registers, memory addresses

4. Tooling: Hand-wiring and punch cards

1950s–60s: Assembly

1. Abstraction Level: Assembly

2. Primary Interaction: Opcode mnemonics

3. Fundamental Requirement: Processor architecture, instruction sets

4. Tooling: Symbolic assemblers

1960s–80s: High-Level Languages (C, Pascal)

1. Abstraction Level: High-Level Programming

2. Primary Interaction: Structured logic

3. Fundamental Requirement: Data structures, pointers, memory management

4. Tooling: Optimizing compilers

1990s–2010s: 4GL / Scripting (Python)

1. Abstraction Level: Higher-level abstractions

2. Primary Interaction: Object-oriented / functional paradigms

3. Fundamental Requirement: Abstraction patterns, libraries, APIs

4. Tooling: IDEs, debuggers, package managers

2020s+: AI-Assisted Programming

1. Abstraction Level: Intent-driven

2. Primary Interaction: Prompt-based

3. Fundamental Requirement: System design, ML, DL

4. Tooling: LLMs, copilots

The fourth and fifth generations have brought us to a point where “coding” is increasingly a task of specifying intent. Engineers now describe the desired outcome, constraints, and context- leaving it to compilers, frameworks, and modern AI systems to plan the exact implementation. Coding is no longer about micromanaging instructions; it is about clearly expressing what should happen and under what conditions.

However, this shift does not eliminate the need for understanding, it raises the bar for it. Just as a Fortran programmer needed to understand the why behind their algorithms to write correct and efficient programs, the modern AI engineer must understand the why behind machine learning systems. Without that grounding, the outputs- whether code or logic- can become brittle, unreliable, or subtly incorrect.

AI systems can generate plausible solutions, but they do not inherently understand correctness or intent. As a result, engineers must be able to reason about failure modes, validate outputs, and recognize when the generated logic is flawed or hallucinated. In this world, specifying intent is powerful - but only when paired with the ability to critically evaluate what is produced.

Final Pillars as you step into this new world of AI with “AI Tools” & “Fundamentals”

Actionable Pillar1:

A few years from now, nobody will remember which model launched this week.

But they will remember the engineers who understood what was actually going on.

Early in your journey, you will feel an urge to keep up and to track every new release, every benchmark, every shiny model that claims to change everything. Resist that instinct. The interface will keep changing, but the underlying principles move slowly. The elite engineers who endure are the ones who understand those principles deeply: how tokens become meaning, how embeddings capture relationships, how optimization shapes intelligence, how architectures like transformers actually work, and how data and compute quietly dictate everything behind the scenes. At last, how thinking, causal reasoning, world modelling, continual learning are still open problems that need breakthroughs.

Also, don’t make the mistake of thinking that theory alone will save you.

Modern AI is not a clean & deterministic discipline anymore. It is messy. It is empirical. It is built as much through experimentation as through understanding. You will often not know why one prompt works and another fails. You will build systems that behave correctly for reasons you cannot fully prove. And that’s okay. Because real engineering lives at the intersection of theory and practice.

First principles will give you compression & reliability in your decision making. (a way to think clearly)

Experimentation (trial&error) will give you truth. (a way to know what actually works.)

Over time, what you are really developing is opinionated taste.

And this becomes even more important as the paradigm shifts from traditional engineering to AI Tools powered engineering.

We are no longer building with models as endpoints. We are building systems where models are just components. The real leverage now lies in how you stitch things together (retrieval systems, tools, memory, orchestration, evaluation loops). For traditional SWEs —> Software Architecture is more important than ever.

Moore’s Law has taught us and has proved that the compute keeps getting cheaper every 18-24 months, and the performance of the computing systems keeps getting doubled every 18-24 months. Hence, the pace of new models, new tools coming up and chips getting cheaper is going to be exponential

But the elite engineers who win are not the ones who wait for better tools. They “learn how to use the current models & tools better than anyone else”.

Actionable Pillar2: Learn to build solid software on these abstractions.

You don’t always need to train models from scratch. You need to know when not to. You will call APIs, compose systems, and build products on top of layers that didn’t exist a few years ago. Just like software evolved from assembly to high-level languages to frameworks, AI is evolving from raw models to primitives to full systems.

Your job is not to go one layer lower or one layer higher blindly.

Your job is to choose the right level of abstraction.

And as you do this, expand your scope. Because it is extremely possible (faster+easier) to expand the scope now more than ever.

Do not stay confined to a narrow role - not as a frontend engineer, not as a backend engineer, not even as an “AI engineer” in the narrow sense. The highest leverage engineers understand the full stack: user experience, backend systems, model behavior, and distributed architecture. They see the system, not just the component.

But there is one final discipline that will separate you from everyone else.

Actionable Pillar3: Evaluation

In this new world, building is easy. You can ship something in a day. But knowing whether it actually works is a question before pushing it out for production.

Making it work Reliably, consistently, under real-world conditions is hard. This is where engineers will have more leverage than no-code based vibe-gineers.

While studying agent reliability, we will also have to learn & push the art of defining evaluation at multiple fronts: datasets/use-cases/scale/functionality/load/stress and study failure modes.

“I am a true artist if I am able to start from first principles, learn best known art, but reimplement it in a way that’s never been done before.”

Lets go back in time: So one of the predictions from uh Geoffrey Hinton(Godfather of AI) who who started the whole deep learning phenomenon. Hinton, An incredible researcher, professor at the University of Toronto, and he invented the idea of back propagation.
Backpropagation allows the neural networks to learn. Hold on to that idea and let us circle back to how people used to write software. Historically, software was written when humans applied first principles thinking. They described an algorithm, a step-by-step approach to do a functionality, and then they codified the whole algorithm or steps or procedures, and the package of the code was called software, simple like a recipe book. Now, getting back to Geoff Hinton. If we think about AI, this invention of artificial intelligence and deep learning, we have to talk about neural networks. Neural networks have a lot of neurons, also simply called math units. You can think of them like a switchboard, and we connect all of these math units together. These math units take in some input, let’s say the image of a cat, and produce or guess what the output might be. So these mathematical units that we described as neurons fire and wire together to produce or guess the output as a cat. All the other signals, let’s say, there is a signal that also guesses a dog, and there is one more signal that guesses an elephant and maybe a tiger. All of these other signals tune or switch to zero when it shows the image as a cat.

Now, this mathematical unit switchboard is very gigantic. The more information you give it, the bigger the switchboard has to be to process the information and guess the output and to be accurate.

The first time when this mathematical unit-based switchboard sees the image of a cat, it produces garbage output. You need to keep on showing it a lot of examples of cats so that there comes a stage where it produces meaningful output, which is called a cat. What Geoff Hinton discovered is the idea of back propagation. While you keep on showing all of these examples of a cat, the output transitions from garbage to guessing a cat nicely. At the back of the envelope, these mathematical units are tuning the signals up and down in order to guess the cat correctly, and they are turning down the signals for dog, for elephant, for tiger.

That is the foundation of artificial intelligence, a piece of software that can learn from given examples. That is basically what we call machine learning, a machine that has the capability to learn. Geoff Hinton discovered this process of learning, which is back propagation. Deep learning is when we move on to the complex switchboards, which is nothing but neural networks.
So one of the first big commercialized applications was image recognition, and the most important image recognition application is radiology. He predicted that about five years ago that in four to five years the world will not need any radiologist.
Because AI would have swept the whole field, but it turns out that AI has swept the whole field and every radiologist is using AI in some way. What is ironic, though, is that the number of radiologists has actually grown. Now the question is why? It’s a very interesting question. Why have the radiologists grown?

The prediction stated that the entire profession of radiologists will be wiped out, but why did we need more.


And the reason for that, if you think deeply, is because the purpose of a radiologist is to diagnose disease, not only to study the image as a task. The image studying is simply one such task in service of diagnosing the disease. Now that we can study more images quickly and more precisely without ever making a mistake, you don’t get tired easily and you can study more images. You can study it in its 3D form instead of 2D, because the AI doesn’t care whether you have given it 3D pixels or 2D pixels. You could also study it in 4D so you can study images better, which a single radiologist could not easily do.

And so the number of tests that people are able to do has increased because they are able to serve more patients. The hospital has just gotten better with more clients, with better radiologists, with more patients. As a result, the overall productivity of the system has increased and produced better economics. When the hospital has better economics, they also tend to hire more radiologists. Because the sole purpose of the job of a radiologist is not only to study images, their purpose is to diagnose a disease.

And so the overarching question that we are leading up to is: what is the purpose of a job? What is the purpose of a doctor? What is the purpose of a lawyer? What is the purpose of a radiologist? Has the purpose really changed?

There is one more question that I would like to introspect. What if my car becomes self-driving? Will all chauffeurs be out of jobs? The answer probably is no, because for some people chauffeurs are a service. For some people, chauffeurs are a part of the hospitality experience. For some people, chauffeurs are there to protect the car owner.
So the Chauffeur that would lose their jobs would be a few, and many Chauffeur would also change their jobs. Now, the ones who are losing it, their sole purpose as a chauffeur was to drive the car for the car owner.

If our job is plain, simple automation, if our job is the task, if our job is only a precise, specific task, then the possibility of losing the job is very high. But if our job is more than a task, just like the shuffle, then we might not get replaced by AI.

We are also seeing what Elon Musk is working on. He is working on machines that make machines and robots. When that happens and when everything goes into production, a whole new industry of technicians and people who have to manufacture these robots will rise. The productivity of Elon’s system will increase, and the number of jobs that never existed will rise up. We are going to have a whole new industry of people taking care of these robots with respect to manufacturing and with respect to maintenance.

Now there would also be some people who would come up with a company that manufactures Apple for robots, because people would want their robot to look different than other people’s robot. This phenomenon will give rise to a whole new industry that never existed.

A mental model for modeling human life using Transformers

One of the most powerful realizations in the last decade of AI is the idea that almost anything can be represented as a sequence of tokens — language, images, code, music, and even protein structures. Once you tokenize something into a sequence, you can train a Transformer to predict the next element of that sequence. That is the foundation of models like GPT.

Today, GPT predicts the next token in a sentence.

Tomorrow, could a model predict the next decision in a human life?

🌍 All of human knowledge is text

The internet holds the collective intelligence of our species — papers, books, conversations, code, opinions, history, news, even emotional expression. We tokenize this massive corpus and train models to learn patterns of how thoughts unfold.

A Transformer receives tokens → embeds them into vectors → and through self-attention learns structure, causality, dependency, and intent. When it predicts the next token, it is essentially predicting human thought continuation.

So here’s the jump:

If text is predictable, and text captures human decisions, then decisions should also be predictable.

🧠 The core idea

What if you take all decisions a single human has ever made:

• Where did I study?

• Which job did I choose?

• Who did I meet?

• What did I buy?

• When did I change habits?

• Which books did I read?

• What content did I consume?

• What goals did I set?

And represent them as ordered tokens in the timeline of life. Now treat this timeline exactly like a sequence of text tokens.

Every decision = a token

Every token has an embedding representing:

• context (why/when)

• emotions

• constraints

• environment

• past experiences

• personality traits

Feed that tokenized life sequence to a decoder-only transformer, and ask:

Given the full sequence of this person’s life decisions so far, what is the most likely next decision?

Autoregressively roll it forward:

Decision1 → Decision2 → Decision3 → ... → Decision_N → predict Decision_(N+1)

You just built a generative model of a human life.

Applications

1. Personal Future Simulation

Simulate your tomorrow, next quarter, or entire decade based on the pattern of your choices.

2. Counterfactual Generators

Ask:

• What would my life look like if I accepted that job offer?

• What if I moved cities?
  You modify the starting token and re-generate the future.

3. Coaching & Self-Awareness

A mirror that shows where your internal algorithm is leading you.

4. Behavioral Optimization

Detect loops and biases:

• impulsiveness

• procrastination

• risk aversion

• pattern of relationships

and propose alternative trajectories.

🔮 The philosophical angle

Humans believe we are infinitely complex and unpredictable — yet we are predictable enough that companies infer our behavior through recommendation models.

But LLMs prove a deeper truth:

Anything that is a sequence can be predicted. And a human life is nothing but a sequence of decisions.

So the idea becomes:

Treat life as a language model.

Treat decisions as tokens.

Treat the future as next-token prediction.

👣 Final thought

We may soon reach a point where we can:

• Upload our history

• Generate multiple future trajectories

• Pick the most meaningful one

Like chess engines compute the best move, life engines could compute the best decision.

The future of AI may not just be generating text.

It may be generating lives.

His idea that some systems don’t just survive volatility but actually get better because of it. And right there, in between loss functions and gradient descent, I stumbled on a single word that connects both worlds: convexity.

Calculus in Our Algorithms and in Our Lives

That beautiful U-shaped curve we love in ML, the convex loss function, is our guarantee of stability.

It tells us:

• Small deviations don’t hurt much.

• Moving in the right direction pays off disproportionately.

• Mathematically elegant. Emotionally… familiar?

Because Taleb’s “convex life” is built on the same logic.

When randomness hits you, the average outcome should improve. You either don’t lose much, or you gain a lot.

The Fragile vs. The Convex Life

The Fragile, Concave Life: smooth, predictable, but one big shock can wipe years of progress.

The Antifragile, Convex Life: messy, volatile, but every small stress adds strength.

You take asymmetric risks, the kind that can quietly fail but might change everything if they succeed.

Applying Convex Optimization to Everyday Life

Career: Most jobs can be linear. The side project is convex. Worst case, you learn. Best case, you take off disproportionately.

Learning: Each concept compounds. Calculus didn’t just help me pass exams, it gave me a framework for life & a framework to take asymmetric bets in investing & in life.

Relationships: Smooth ones can be fragile. Convex ones embrace honest stress, emerging stronger after each conflict.

Convexity, it turns out, isn’t just a mathematical property. It’s a way of structuring your existence.

Convex optimization makes algorithms stable.

Convex living makes humans antifragile.

And maybe, just maybe, that’s the most practical use of calculus we’ll ever find outside a classroom.

Subscribe now