Reading math books

[fluffy]

Math is a huge subject.

It is like reading.

When you read there are thousands of words that can go together that mean different things.

But it is done differently in that the abstractions used are about relationships in space and quantities where meaning is less intuitive or emotional action based.

I barely have been understanding vectors.

They are in part used on top of each other to see if two shapes are invariant (a triangle is a triangle if it is big or small) in n dimensional spaces.

In statistics you are trying to predict things happening where you have sparse data, spurious correlations, and if the relationship of x to y in a slope fits for the data points accurately.

Curves for the same reasons go x to y

These and other such maths were done by hand at first so arithmetic was what you needed first.

Making shapes with formula.

Computers could do it faster but you still needed to put the data and algorithms together.

Negative space as the square root of -1

That helps you to know if you can infer wave patterns.

Wave patterns are complex ways to make predictions.

Like in electrical engineering.

Or economic supply chains.

-

I have ideas about all that.

Not sure how I could implement it.

I will not be able to program it. But I could probably pay someone if I get money.

What I need to do is learn more maths.

The book I have is a graduates school book.

Since I read the first chapter I don't know how but I understand it, might be because I was doing other math problems in a history book about curves and probability for 6th graders.

 

[fluffy]

I have a hard time with numbers.

I cannot add or multiply fast.

This is why I think they did not put me in calculus classes.

But I was really good at spatial tasks.

I can do Mensa level puzzles.

-

I got a math book you need to learn all the maths in it to get a masters degree.

I understand it.

I have had brain fog a long time but recently it cleared up, as I said they did not have me in classes for higher maths because of my inability to calculate numbers. They did not give me books to read about math even. So looking at this new book I really like it.

I suppose that you can go beyond this kind of math as well. If it is this simple to understand for people that are 30 when the graduate then imagine what they do for 20 years afterwards using these abstract tools.

Some people spend their entire lives making up math tools so a masters degree in math is just the beginning.

Labs that use math on supercomputers.

They probably discover new maths all the time.

(I still have ideas for self assembling networks I want to try out on a computer)

 

[dr froyd]

learning math is a great way to learn how to think clearly, especially when it comes to computation, programming stuff

 

[birdsnestfern]

AI grokking (no idea how to understand it, but it’s when an Ai system slows down as it suddenly Gets something new It’s learning). If it asks you to join use GitHub email, or create a new GitHub account so you can see the stream.

https://arxiv.org/pdf/2501.04697?

Grokking at the Edge of Numerical Stability

Lucas Prieto, Melih Barsbey, Pedro A.M. Mediano, Tolga Birdal

Imperial College London 2025
https://arxiv.org/abs/2501.04697

Grokking at the Edge: How Numerical Instability Shapes Learning in AI

Deep learning can be unpredictable. One of the strangest phenomena is grokking—when an AI model seems to be stuck memorizing data for a long time, only to suddenly generalize and understand patterns much later. But why does this happen? And why does regularization (a technique that prevents overfitting) play such a key role?

A new study suggests that, without regularization, grokking forces AI models into dangerous territory: the edge of numerical stability. This means that as training continues, tiny errors creep into the Softmax function—the part of the network that helps it make decisions. The researchers call this Softmax Collapse (SC), and they find that once SC happens, learning stops altogether, preventing grokking.

So what’s causing this collapse? It turns out that past a certain point, the model’s learning process gets stuck in a feedback loop. Instead of improving its ability to generalize, it just scales its internal weights in a way that reduces loss mathematically but doesn’t actually refine its predictions. This misalignment explains why grokking takes so long and why it sometimes never happens at all.

To tackle this problem, the researchers introduce two innovations:

1. StableMax, a new activation function that prevents Softmax Collapse, allowing grokking to occur even without regularization.

2. ⊥Grad, a new training algorithm that forces the model to focus on meaningful learning rather than just minimizing loss in an unhelpful way.

These findings provide fresh insights into grokking and could lead to more stable and efficient AI training techniques. In short, the mystery of grokking may finally have a numerical explanation—and a potential fix.

Notes:

What Is Grokking in AI?

Imagine training a student for a math test. At first, they memorize the answers without really understanding the concepts. But then, after weeks of practice—even without seeing new material—they suddenly have a breakthrough, grasping the deeper logic behind the problems.

This mysterious phenomenon happens in AI too, and researchers call it grokking. It was first discovered in deep learning models trained on simple tasks. At first, the model just memorizes the training data, making no real progress in generalizing to new examples. Then, after a long period of seemingly pointless training, it suddenly figures out the underlying pattern and generalizes perfectly.

Why Is Grokking Surprising?

Normally, AI models either learn to generalize early or overfit, meaning they memorize data without developing real understanding. Grokking defies this expectation: generalization happens, but only after prolonged overfitting. This challenges conventional wisdom in deep learning and raises big questions:

What causes the delay in generalization?

Why does grokking happen in some cases but not others?

How do factors like regularization (which prevents overfitting) influence grokking?

Why Does Grokking Matter?

Understanding grokking could lead to breakthroughs in AI training. If we can control and optimize it, we might build models that generalize better, require less data, and avoid the pitfalls of overfitting. It could also shed light on how learning works—not just in machines but possibly in human cognition too.

Current Research on Grokking

Scientists are still working to unravel the exact mechanisms behind grokking. Some recent studies suggest that grokking happens when certain learning patterns align in just the right way, while others propose that it’s tied to how models optimize their internal representations over time. The latest research even links it to numerical stability issues in AI, meaning small computational errors could be influencing the process.

In short, grokking is one of the strangest and most exciting mysteries in deep learning—one that could reshape how we train AI in the future.

#AI #machinelearning #computervision #patternrecognition

Join us for an exclusive discussion on Grokking!

Why does AI sometimes take forever to generalize, only to suddenly "get it"? The mystery of grokking is pushing the boundaries of deep learning, and we’re diving deep into the latest research!

Speaker: Prof. Tolga Birdal (Imperial College London)
Host: Cecile G. Tamura, Head of Community, Ploutos

Date: March 21
⏰ Time: 7 a.m. PST / 10 a.m. EST
Join here: https://app.ploutos.dev/streams/abstract-corgi

We’ll explore how numerical stability issues can block AI from achieving grokking and how new techniques can unlock better generalization. If you're into AI, machine learning, or just love a good intellectual mystery—this is a talk you don’t want to miss!

 

[birdsnestfern]

Average of every piece of land in a state, ie, some land isn’t usable so avg might be less than the actual price in general.

Average cost per acre of land in all 50 states:

• Alabama: Approximately $18,103 per acre
• Alaska: Approximately $1,300 per acre
• Arizona: Approximately $4,200 per acre
• Arkansas: Approximately $11,596 per acre
• California: Approximately $56,000 per acre
• Colorado: Approximately $11,561 per acre
• Connecticut: Approximately $73,000 per acre
• Delaware: Approximately $18,000 per acre
• Florida: Approximately $19,300 per acre
• Georgia: Approximately $8,200 per acre
• Hawaii: Approximately $27,000 per acre
• Idaho: Approximately $10,500 per acre
• Illinois: Approximately $9,000 per acre
• Indiana: Approximately $7,000 per acre
• Iowa: Approximately $9,500 per acre
• Kansas: Approximately $2,800 per acre
• Kentucky: Approximately $6,400 per acre
• Louisiana: Approximately $5,800 per acre
• Maine: Approximately $10,500 per acre
• Maryland: Approximately $36,000 per acre
• Massachusetts: Approximately $98,000 per acre
• Michigan: Approximately $18,333 per acre
• Minnesota: Approximately $6,700 per acre
• Mississippi: Approximately $10,835 per acre
• Missouri: Approximately $14,078 per acre
• Montana: Approximately $10,000 per acre
• Nebraska: Approximately $6,000 per acre
• Nevada: Approximately $3,000 per acre
• New Hampshire: Approximately $18,000 per acre
• New Jersey: Approximately $90,000 per acre
• New Mexico: Approximately $6,000 per acre
• New York: Approximately $12,027 per acre
• North Carolina: Approximately $7,000 per acre
• North Dakota: Approximately $3,200 per acre
• Ohio: Approximately $7,500 per acre
• Oklahoma: Approximately $19,628 per acre
• Oregon: Approximately $16,162 per acre
• Pennsylvania: Approximately $11,000 per acre
• Rhode Island: Approximately $350,400 per acre
• South Carolina: Approximately $6,800 per acre
• South Dakota: Approximately $2,800 per acre
• Tennessee: Approximately $8,000 per acre
• Texas: Approximately $4,500 per acre
• Utah: Approximately $195,000 per acre
• Vermont: Approximately $12,000 per acre
• Virginia: Approximately $8,500 per acre
• Washington: Approximately $15,000 per acre
• West Virginia: Approximately $6,200 per acre
• Wisconsin: Approximately $7,800 per acre
• Wyoming: Approximately $2,500 per acre