(Black Swan Theory) Please Clarify for me

[menaceh2k]

I am afraid that this post will expose the limits of my intelligence but I have to do it none the less. I've read Nassim nochola's Black Swan:

http://en.wikipedia.org/wiki/Black_swan_theory

Spoiler

The Black Swan Theory or "Theory of Black Swan Events" was developed by Nassim Nicholas Taleb to explain: 1) the disproportionate role of high-impact, hard to predict, and rare events that are beyond the realm of normal expectations in history, science, finance and technology, 2) the non-computability of the probability of the consequential rare events using scientific methods (owing to their very nature of small probabilities) and 3) the psychological biases that make people individually and collectively blind to uncertainty and unaware of the massive role of the rare event in historical affairs. Unlike the earlier philosophical "black swan problem", the "Black Swan Theory" (capitalized) refers only to unexpected events of large magnitude and consequence and their dominant role in history. Such events, considered extreme outliers, collectively play vastly larger roles than regular occurrences.

and I do not understand anything beyond there are things that are difficult-impossible to predict because we are accustomed to perceiving certain things a certain way. And my favorite part...you cant do a damn thing about it...


I mean i read the book front to back, and I liked it. But I did not get anything more out of it than what i posted above. Maybe its because i got a shameful B- in statistics. I know I know. As far as I'm concerned statistics is not real MATH. Its bullshit. Its dirty nasty and unordered. Ewwwwwww. So anyways, did he write a million page book just to repeat the same thing over and over again. Or I'm I missing something? Is my hatred for dirty statistics (the math of charlatans) blinding me to the answers.

Is it even a theory. It reminds me of the whole notion of "there are known knowns and unknown unknowns." which is also bullshit. and "unknown unknown" is just an unknown. yet i digress. please help me get out of this stupid mental loop and clarify this for me.

 

[EyeSeeCold]

lol. It's not even that complicated. And it's not a theory:

http://en.wikipedia.org/wiki/The_problem_of_induction#David_Hume

For example, when one thinks of "a billiard ball moving in a straight line toward another,"[7] one can conceive that the first ball bounces back with the second ball remaining at rest, the first ball stops and the second ball moves, or the first ball jumps over the second, etc. There is no reason to conclude any of these possibilities over the others. Only through previous observation can it be predicted, inductively, what will actually happen with the balls. In general, it is not necessary that causal relation in the future resemble causal relations in the past, as it is always conceivable otherwise; for Hume, this is because the negation of the claim does not lead to a contradiction.

Basically, you can't truly predict or know what will happen or what is out there based on previous empirical or rational observation because the past does not necessarily determine the future.

 

[BigApplePi]

I am afraid that this post will expose the limits of my intelligence but I have to do it none the less. I've read Nassim nochola's Black Swan:

http://en.wikipedia.org/wiki/Black_swan_theory

Spoiler

The Black Swan Theory or "Theory of Black Swan Events" was developed by Nassim Nicholas Taleb to explain: 1) the disproportionate role of high-impact, hard to predict, and rare events that are beyond the realm of normal expectations in history, science, finance and technology, 2) the non-computability of the probability of the consequential rare events using scientific methods (owing to their very nature of small probabilities) and 3) the psychological biases that make people individually and collectively blind to uncertainty and unaware of the massive role of the rare event in historical affairs. Unlike the earlier philosophical "black swan problem", the "Black Swan Theory" (capitalized) refers only to unexpected events of large magnitude and consequence and their dominant role in history. Such events, considered extreme outliers, collectively play vastly larger roles than regular occurrences.

and I do not understand anything beyond there are things that are difficult-impossible to predict because we are accustomed to perceiving certain things a certain way. And my favorite part...you cant do a damn thing about it...


I mean i read the book front to back, and I liked it. But I did not get anything more out of it than what i posted above. Maybe its because i got a shameful B- in statistics. I know I know. As far as I'm concerned statistics is not real MATH. Its bullshit. Its dirty nasty and unordered. Ewwwwwww. So anyways, did he write a million page book just to repeat the same thing over and over again. Or I'm I missing something? Is my hatred for dirty statistics (the math of charlatans) blinding me to the answers.

Is it even a theory. It reminds me of the whole notion of "there are known knowns and unknown unknowns." which is also bullshit. and "unknown unknown" is just an unknown. yet i digress. please help me get out of this stupid mental loop and clarify this for me.

I read half the other book by Taleb. I forget its name. Thoughts:

1. Black Swans (if they can happen at all) have a high probability of happening if you wait long enough.

2. They have a big impact because you tend to notice them because they are unexpected.

3. Here is an example: Every day. Every single day you wake up in the morning to discover you are alive. Pretty soon that is routine and you fearlessly go to sleep expecting to wake up. That is a mistake. One day you will go to sleep and will suprised to wake up dead because no one lives forever. That will not only be a bad day for you, but will be a black swan day for you as well.

4. What are "dirty" statistics? I like all of them. Are you biased?

 

[Infinite Regress]

I read half the other book by Taleb. I forget its name.

Fooled by randomness

And my favorite part...you cant do a damn thing about it...

He does. In a trading context, his portfolio is net long gamma/convexity, taking advantage of black swans // leptokurtosis in asset returns.

 

[Vrecknidj]

Personally, I tend to remember really embarrassing moments in my life far more vividly and for far longer than just about any other memories. My own personal Black Swans?

I agree with the 10 principles.

Dave

 

[BigApplePi]

Personally, I tend to remember really embarrassing moments in my life far more vividly and for far longer than just about any other memories. My own personal Black Swans?
Dave

I was going to say, "Me too", but now I'm not so sure. My socially embarrassing moments (which I remember as you do) are usually preceded by fear. Doesn't that mean I knew something bad was likely to happen because I didn't know the socially correct thing?

 

[Jesse]

Statistics are based on an illogical principle in the first place, although they can help discover what will happen quite well. Can any one name any Black Swan's in history. With 20/20 hindsight I bet they can all be explained by what preceded it.

 

[BigApplePi]

Statistics are based on an illogical principle in the first place, although they can help discover what will happen quite well. Can any one name any Black Swan's in history. With 20/20 hindsight I bet they can all be explained by what preceded it.

"Illogical" is a tough word here. I think statistics is about the unknown. When we don't know how a coin will go flipped, the odds are 50-50 because we are ignorant. If we had all the details of physics on the coin: starting position, angular momentum, wind conditions, etc., we could predict how the coin would go, but we don't.

Statistics are about measuring ignorance and knowledge. We know the coin will go only one of two ways. We are ignorant of any inclination of which way, so we call them equal.

Black Swan's in history? Pick a person deciding to sail on the Titanic. Pick John Kennedy's assassination. There's a black swan. The shooter could have missed and history today would have been different. I can't be sure of that though. Maybe a better example in history can be thought of.

 

[Melllvar]

"Illogical" is a tough word here. I think statistics is about the unknown. When we don't know how a coin will go flipped, the odds are 50-50 because we are ignorant. If we had all the details of physics on the coin: starting position, angular momentum, wind conditions, etc., we could predict how the coin would go, but we don't.

Statistics are about measuring ignorance and knowledge. We know the coin will go only one of two ways. We are ignorant of any inclination of which way, so we call them equal.

Isn't that just a misuse of the word "statistical" though? If you don't know enough about the situation to determine what will happen it doesn't default to some statistical number, it just means your numbers are wrong. As an example, if, in an election, you do extensive polling and 9/10 people claim they will vote for candidate A, and I don't do the polling and have no idea who is even running, the chances of candidate A winning aren't actually different between us. It's not really a statistical situation, we're just using an analogy to describe our guesses. There is no chance involved.

Edit: I suppose this is just semantics though. You're using the word to describe your guess of the situation, while I'm using it to describe the chances of truly random events.

 

[BigApplePi]

Isn't that just a misuse of the word "statistical" though? If you don't know enough about the situation to determine what will happen it doesn't default to some statistical number, it just means your numbers are wrong. As an example, if, in an election, you do extensive polling and 9/10 people claim they will vote for candidate A, and I don't do the polling and have no idea who is even running, the chances of candidate A winning aren't actually different between us. It's not really a statistical situation, we're just using an analogy to describe our guesses. There is no chance involved.

Edit: I suppose this is just semantics though. You're using the word to describe your guess of the situation, while I'm using it to describe the chances of truly random events.

I lost ya Melllvar. I wasn't making a formal definition of "statistics" or even trying. Not sure I follow you. In one case I do polling and in the other you guess. Because I did polling I have more information. Therefore won't my judgment as to the odds for the outcome be closer to reality? Statistics is not about the reality of an event actually as it occurs. It's about judgment before it occurs.

 

[Melllvar]

Er, I think you're probably right actually, the more I think about it, I just don't like the idea that a statistic is based in someone's ignorance. It seems like there are "correct" statistics/probabilities and "incorrect" ones. Just because I'm totally ignorant of a situation doesn't mean the actual probability is 50/50, it just means I don't know what the real probability is.

The problem with that though is that it would imply no deterministic situation could accurately be described by statistics (since the outcome would always have a likelihood of either 100% or 0%). If we say a coin has a 50/50 chance of being heads, we're just guessing and citing those numbers to demonstrate ignorance, and most situations are deterministic, so by my previous logic it would mean pretty much every time someone referred to a statistical measure they would be misusing the word. At which point it became obvious that I was just using a different definition of the word (while a coin toss doesn't actually have a 50/50 chance of happening either way, we just describe our knowledge of the situation using those numbers).

That might not have been much clearer, sorry, I think I'm more explaining this to myself than to anyone else now.

 

[BigApplePi]

Er, I think you're probably right actually, the more I think about it, I just don't like the idea that a statistic is based in someone's ignorance. It seems like there are "correct" statistics/probabilities and "incorrect" ones. Just because I'm totally ignorant of a situation doesn't mean the actual probability is 50/50, it just means I don't know what the real probability is.

The problem with that though is that it would imply no deterministic situation could accurately be described by statistics (since the outcome would always have a likelihood of either 100% or 0%). If we say a coin has a 50/50 chance of being heads, we're just guessing and citing those numbers to demonstrate ignorance, and most situations are deterministic, so by my previous logic it would mean pretty much every time someone referred to a statistical measure they would be misusing the word. At which point it became obvious that I was just using a different definition of the word (while a coin toss doesn't actually have a 50/50 chance of happening either way, we just describe our knowledge of the situation using those numbers).

That might not have been much clearer, sorry, I think I'm more explaining this to myself than to anyone else now.

I think it great to think about these issues. It's okay to search out the meaning of things.

Note: I wasn't thinking of ignorance as being a shameful or negative thing at all. We are always ignorant of something in a situation and are uncertain. These words, "statistics", "chance", "probability" do require definitions. I guess we have to define what it means for an unbiased coin to have a 50/50 probability of heads.

 

[Infinite Regress]

We are ignorant of any inclination of which way, so we call them equal.

Doesn't one only assume equal probability if events [coin flip] are independent of each other?

 

[BigApplePi]

Doesn't one only assume equal probability if events [coin flip] are independent of each other?

You said the magic word, "events." A coin flip is only one event.

 

[Infinite Regress]

You said the magic word, "events." A coin flip is only one event.

I thought probability referred to proportion of outcomes over the long run, rather than one flip of the coin?

 

[BigApplePi]

I thought probability referred to proportion of outcomes over the long run, rather than one flip of the coin?

Yes. Not sure what we were talking about. Events must be independent. What happens to the 2nd coin flip is independent of what happens to the 1st coin flip. One can get ten heads in a row and the probability of the 11th going one way or the other is still equal. The probabilities are equal not because of the past but because there is no bias.

Now if I were to ask, what is more probable: that someone's eyes are blue or that another one weighs less than 145 pounds, I assume those events are independent. That doesn't make them equal though. They are not equal because they are different things. A coin flip heads or tails is mutually exclusive.