Inverted Passion

Know what's true and do what's right

All posts in Philosophy

LLM are universal simulators

Posted on Author Paras ChopraPosted in Philosophy, Technology

Saying that LLMs are just next token predictors is underselling these beasts to a mind numbing degree.

First, LLMs aren’t just predicting the next token. They plan ahead – the loss function is the average of cross entropy across all future tokens in a context window and the attention has access to all previous tokens. So, at a particular token, the LLM is planning what could be relevant far ahead and not just at the immediate next token.

Second, LLMs are trained to predict sequences across all texts on the internet that contain not just human generated text but things like weather forecasts, financial series, code, bash dumps, satellite pings and so on. To be able to do this prediction well, LLMs have to infer the physics / dynamics for all such domains (e.g. to predict weather patterns in data, you need to develop a model of earth coordinates, sunlight patterns, monsoon cycles and so on). ...  Read the entire post →

Why math works so well in describing our universe?

Posted on Author Paras ChopraPosted in Philosophy, Physics

Wigner wondered about the unreasonable effectiveness of mathematics in describing our universe.

That is, why are our physical laws as described by precise mathematics produce predictions that turn out to be true?

Let’s look at it from three perspectives.

#1 We grasp the graspable bits

By definition, we model aspects of reality that we are capable of modeling. The remaining part may be truly random or complicated. For example, where do quantum measurements come from? Or the incompatibility between general relativity or quantum mechanics. ...  Read the entire post →

Why is there something rather than nothing?

Posted on Author Paras ChopraPosted in Philosophy

Brace yourself for some wild speculations!

Been thinking if nothing is even possible.

I think it’s impossible for nothing to exist.

When we’re sleeping, it’s not as if we’re feeling nothing. There’s actually no feeling then. It’s as if such moments don’t even exist, and that’s a hint..

So the more I think, the harder it’s for me to imagine nothing existing. Can you really imagine no concepts existing? No numbers, no space, no time, no blip – just nothing at all. If you can imagine that, something (the imaginer) must exist. ...  Read the entire post →

Life as a physical process

Posted on Author Paras ChopraPosted in Biology, Book Summary, Philosophy, Physics

It’s always hard to define life. Everyone has their favorite definition – some describe it as a struggle against entropy, while others describe it as an emergent property of chemicals. Countless books have been written on the topic, yet we’re far from a consensus.

Against the backdrop of the second law of thermodynamics, life seems like an improbable accident. When everything tends to go towards disorder, how come life is able to create cities, computers and space ships? How do we reconcile all the beautiful complexity we see around us with the stupidly simple laws we observe in physics? ...  Read the entire post →

Don’t compete

Posted on Author Paras ChopraPosted in Personal, Philosophy

The Internet is full of people winning all the time. Someone is traveling to exotic locations, someone else is raising funds, and another person is winning awards. Essentially, everyone around you is succeeding while you do spend your days as the nature intended – sleeping, eating, smiling, chatting with friends, and spending time with your cat.

But, who here is really winning?

Imagine our society as a living and breathing organism with its own agenda. What would be on its agenda? First of all, like you, society would want to not die. Second, like you, it would want to thrive. ...  Read the entire post →

What bootstraps intelligence?

Posted on Author Paras ChopraPosted in Philosophy, Systems

A musing on how intelligence comes to be.

The bedrock of intelligence is abstractions – the thing we do when we throw away a lot of information and just emphasise on a subset of it (e.g. calling that thing an apple instead of describing all its atoms and their x, y, z positions).

But where does the drive to form abstractions comes from? What if it rose from our desire to communicate with others? Since communication bandwidth is always limited, we are driven to find most efficient way of getting an idea across which leads to abstractions. Imagine a world where energy and time is unlimited, we might be communicating all x,y,z positions of things instead of putting labels on them. ...  Read the entire post →

Not everything is physics

Posted on Author Paras ChopraPosted in Philosophy, Physics

The first book I ever read was The Brief History of Time by Stephen Hawking. I liked it so much that I re-read it 8 times. As a young boy, the book had made a lasting impression on me, making me fall in love with ideas such as the arrow of time, black holes, entropy, and Big Bang. Reading this book, you can’t help but open up to the spirit of science that pushes you to keep exploring the boundaries of knowledge, one hypothesis at a time.

I am very much a product of such thinking process. In fact, during one of my sabbaticals, I took up the goal to understand all of currently unsolved problems in physics. This required me to brush up on quantum mechanics, general relativity, cosmology, and standard model. It took some serious effort, but in the end, I’m glad I was able to rise up beyond the pop-science level of understanding of physics. By the end of my sabbatical, I was finally able to look at quantum mechanics equations and understand what they were about. ...  Read the entire post →

Usefulness grounds truth

Posted on Author Paras ChopraPosted in Mental Models, Philosophy

Are LLMs intelligent?

Debates on this question often, but not always, devolve into debates on what LLMs can or cannot do. To a limited extent, the original question is useful because it creates an opening for people to go into specifics. But, beyond that initial use, the question quickly empties itself because (obviously) the answer to the question if X is intelligence depends on how you define intelligence (and how you define X).

Even though it is clear that words are inherently empty, internet is full of such debates. People focus on syntax, when semantics is what runs the world. ...  Read the entire post →

You can’t jail an AI

Posted on Author Paras ChopraPosted in Philosophy

Here’s why I worry about AI.

We know that people can get away with anything to pursue their goals (of profit, power, etc.) as long as they know they can get away with it, without negative consequences. We have had Hitlers, and insider traders.

But the world keeps them in check via law and guns.

Like humans, AIs will have goals (like maximize profit or please a human via an entertaining chat) and they will be cleverer than humans in coming up with schemes that help them get away with their plans without negative consequences. ...  Read the entire post →

Why time seems to pass faster as we age

Posted on Author Paras ChopraPosted in Personal, Philosophy

1/ I’ve been mega-obsessed with this feeling.

A year as a 36-year-old seems so much shorter as compared to when I was a kid or even as a teen.

It seems cosmically unfair – we have fewer years to live, and each year flies by faster.

2/ But, why is that happening?

My tentative conclusion is that it’s an unfortunate outcome of how evolution shaped our brain to be an efficient storage device. 

3/ Our brain is a prediction device.

Its top job is to construct a model of the world so that we get a survival and reproductive edge.  ...  Read the entire post →