Who would have thought that so many people in the business world apparently don't understand basic probability statistics?

The media has commonly stated that, prior to 2007, most people assumed that American home prices would continue to rise indefinitely, due to the fact they had never once fallen, on a national level, during 50+ years (although, to be fair, I don't recall particular executives being singled out).

Now, a 50 year history of the housing market may sound like a long time. It's actually a very SHORT time.

If you measure 50 years in terms of seconds (1,576,800,000) or minutes (26,280,000) or even days (7,300), the sample size is moderate to large.

I would argue that many people (who were otherwise capable enough) who made decisions based on the 50+ year housing history were actually making decisions based on the sample's number of years, not days, minutes or seconds. This is where they erred.

If they were basing it on days, they never could have made the claim that the national housing market prices never fell from one day to another during the entire 50-year history. They never could have claimed that the national housing prices never fell from one second to the next.

So, they were making a claim based on a sample size of only 50.  That is not even close to being a significant enough of a sample size for making ideal predictions (and anyway, such a prediction would need to be adjusted by accounting for the context of the past 50 years and the expected context of the future; for example, home prices in the past could have been inflated, relative to future expectations, due to the baby boom, which resulted in a larger than usual number of people buying homes during a certain decade (when the boomers were reaching their 20s and 30s).

Sometimes even very large sample sizes aren't enough to provide an accurate snapshot of actual expectations.  When it comes to blackjack, if you play basic strategy, which results in a loss of about 0.5% over time, one simulation I saw showed that you can run 10 different sets of 100,000 games, and during one of those sets you will actually win money, even though the strategy used is a losing strategy!

So, at times even 100,000 isn't a large enough of a sample size to properly reflect long term expectations.  Yet, supposed experts apparently believed that a sample size of 50 was enough to make judgments about the housing market.  Does this frighten anyone?  Perhaps I should be hired to consult the "experts" (note that the blackjack example is rarer than others, because blackjack has a very high standard deviation. Still, a sample of 50 is extremely low by just about any standards).

Think about how tiny a sample size of 50 is.  You could run a simulation of coin flips, and it would be quite common to find sets of 50 flips that had results which deviated quite a bit from the expected long term reality (25 heads, 25 tails).  You would see sets of 30H/20T, 26H/24T etc.

Now, it is true that during university I earned a grade of 100% (actually >100% when you include the bonus question) on an exam testing probability (finite mathematics), so I do have an advantage over most.  But were none of the executives and quants brainiacs regarding probability?

Maybe executives actually did understand probability statistics but had a problem with intelligence and perception: perhaps they actually did believe their sample size was huge. They thought in seconds instead of years.

Who would have thunk it?

Alice O. Nunez
10/26/2010 08:02:05 am

"But were none of the executives and quants brainiacs regarding probability?"

I'm a combinatorist, but I won't pretend that my facility with probability is sufficient for understanding housing markets. You're making a ton of ridiculous assumptions here which no economist would make:

1) That the language of sample size and standard deviation is correct to apply to a time series, in which the data aren't independent.

2) That short-term day-to-day fluctuations are better than year-to-year comparisons at explaining long-term trends.

3) That economists look at only one data point per year when quantifying anything.

4) That a sample size of fifty can't tell you anything. Of course it can—if someone hands you a supposedly unbiased coin, and you proceed to flip fifty heads on your first fifty flips, there's a pretty good case that they're putting you on.

And, like every other post on your blog,

5) That nobody else has ever thought of all of this before.

Reply
11/2/2010 02:18:50 pm

Alice,

incorrect. I just noticed this past post of yours. I now recall why I've banned you from making future, apparently troublemaking, posts on this board.

1) A time series does include independent data. Each second is independent from the next, and measures the same. If that were NOT the case, it would not be possible to measure time with clocks.

2)I never said that short term trends were better than yearly trends as explaining long term trends.

I said that long term trends WERE the better way to predict, and the problem is that people defined the long term using a small sample size of years instead of using a large sample size of minutes or days.

3)I never claimed economists look only at one data point.

4)I never said a sample size of fifty can't tell you anything. I said that a larger sample size is better, and that a smaller sample size is more likely to be less representative.

Reply
Anonymous
11/11/2010 03:40:09 pm

Oh my god you are so stupid. You explain that 50 years isn't a long time by converting it to seconds? Are you bloody high? Do you understand even the most fundamental aspects of statistics?

Reply
11/11/2010 03:40:26 pm

Anonymous,

sadly you don't seem to understand that a sample size of 50 isn't very significant. It's you that doesn't understand fundamentals of stats and sample size.

Reply
Reader 80
11/11/2010 03:40:46 pm

The media has commonly stated that, prior to 2007, most people assumed that American home prices would continue to rise indefinitely, due to the fact they had never once fallen, on a national level, during 50+ years. (Although, to be fair, I don't recall particular executives being singled out).

you must agree that if the media gave this info to the public in the term of seconds or minutes they would be like WTF.... "let me go get a calculator, not"

I would like to believe that they would look at this in a bigger fraction (at least in days or weeks).

Let me try to simplify what I think you are trying to explain *NSTAAO* say you have 500 sheep and 3 are pink.... If you were to use a complete profiled fraction you would see that 497/500 sheep are white.. but if you made that into a small fraction and rounded it... You could easily make it into 50/50... Much like they did with the housing market. So say for 7200/7300 days the housing market was stable. Put that in terms of years and round it out and poof you have a perfect 50/50 which is technically inaccurate.

But, like I stated before I think the media just used the smaller fraction to simplify it to the general public. I hope they would use a bigger profile to understand something as complex as housing markets.
Now why is ratio size important???
7200/7300 days is still a perfect 50/50 years until you hit 7117/7300 that gives a good 183 days (1/2 a year) leeway in which the market can constantly fall.
Also if you were to measure the housing market as say 10 at year 1- year 2 = 20 year 3= 39 year 4 = 46 year 5 = 53 and so on..... Yes the housing market is still going to be increasing for 10 years but if you were to look at a bigger scale (days) there would be growing and falling lines on the graph rather than just growing like in a year graph. So you would notice that the falling lines are getting longer and steeper and the rising lines are getting shorter and flatter. This would allow you to see that the housing market is falling rather than growing.

Reply
Anonymous
11/11/2010 03:41:04 pm

You've fundamentally misunderstood the notion of sample size (and also of statistical significance, it seems).

Sample size refers to the number of unique observations in the data. In this case, that would be house transactions. For each year, then, the sample size would be the number of transactions that have been completed in that year. You would be comparing maybe hundreds of thousands of transactions between years.

Reply
11/11/2010 03:41:41 pm

Reader 80,

you write:

"you must agree that if the media gave this info to the public in the term of seconds or minutes they would be like WTF.... "let me go get a calculator, not""

With respect: who cares? First, there was no need to provide the information in seconds at all, since their being aware of the "seconds" sample size meant they would be aware that housing wouldn't go up, and hence they should have simply said that there is no reason to believe that housing prices will continue going up indefinitely. They don't need to explain the reasoning (sample size in seconds) behind that.

And I would argue that if they were forced to explain the sample size of seconds, resulting in confusion, that's much more preferable to presentation of an unconfusing, but wrong, thesis: that housing will rise based on a sample size of 50.

You write:

"If you were to use a complete profiled fraction you would see that 497/500 sheep are white.. but if you made that into a small fraction and rounded it... You could easily make it into 50/50... "

False. They used the figure of 50 because at the end of each year all 50 years had home price rises. There is no reason to believe otherwise, so it should be assumed that they meant what they said, no?

Another problem with your argument is this:

You are applying two different sets of criteria to the same figures. You are choosing to round not to round 497/500 up to 500/500 in the first case, and then choosing to round 49.7/50 up to 50/50. But might someone do that? Sure, but that doesn't validate an argument that uses two sets of criteria.

But I think I know what you're attempting to say, and I commend you for your attempt.

You continue:

"Much like they did with the housing market. So say for 7200/7300 days the housing market was stable. Put that in terms of years and round it out and poof you have a perfect 50/50 which is technically inaccurate."

A very poor example. First, being "stable" for 7200 out of 7300 days is not equivalent to a "rise". Stable could mean slightly higher or slightly lower, and knowing nothing else, it would mean "lower" as often as "higher", to balance out as "stable".

Also, to be able to plausibly round up x/7300 to 7300 out of 7300, x must be a very high number.

I highly doubt that home price rises in a magnitude close to as as many as 7200 out of 7300 days.

I would think that during some calendar years, for some periods (several months of a year) prices were down since Jan 1, but then rebounded sharply by Dec 31 to count as an up year.

You write:

"I think the media just used the smaller fraction to simplify it to the general public. "

Yes, had the media received the larger fractions, the media might not have used the larger denominators, but as I mentioned, I doubt anyone even presented the fractions with the larger denominators (in seconds or minutes or days) to the media.

Why? Well, to present the larger denominators to the media, they would've had to be aware of it. If the people presenting the info to the media were aware of the larger denominators, they wouldn't have been making the claim that the housing market wouldn't ever fall again.

Reply
11/11/2010 03:42:20 pm

Anonymous,

you write:

"Sample size refers to the number of unique observations in the data. In this case, that would be house transactions. For each year, then, the sample size would be the number of transactions that have been completed in that year. You would be comparing maybe hundreds of thousands of transactions between years. "

That's a bizarre statement. Did you even read my post?

My entire premise was exactly what you imply:

That the sample size being analysed shouldn't be as small as 50, because there are so many home transactions and rises and falls within those 50 years that the sample size should be much larger, as measured in seconds (or, as you suggest, by the actual number of home transactions).

Reply
Anonymous
11/11/2010 03:42:38 pm

I'm sorry, but you're spouting nonsense about a subject you clearly don't understand. This is dmonstrated by the fact that you say "So, they were making a claim based on a sample size of only 50. That is not even close to being a significant enough of a sample size for making predictions." This statement betrays a lack of understanding about significance testing. It is not sample sizes that are significant or not, but rather the shifts observed between data.

To reiterate - one cannot chose how to calculate sample size. It is an absolute, dependent on the number of unique observations. Sample size is completely independent of time, except insofar as each year, a particular number of housing transactions will occur. This number represents the sample size for that year.

Each year therefore represents a subset of the data, similarly to how a piece of research might analyse data by ethnicity.

If, as you suggest, house prices were to be analysed by the day, then the sample size for each subset would be drastically smaller, since fewer transactions occur each day. This would therefore mean far lower likelihood of seeing any significant shifts in the data.

The analysis you initially critique is completely legitimate. You have simply misunderstood it.

Reply
Anonymous
11/11/2010 03:47:53 pm

Anonymous,

let me school you:

You write:

"This statement betrays a lack of understanding about significance testing. It is not sample sizes that are significant or not, but rather the shifts observed between data."

It's true that smaller samples sizes can be significant, but it's always true that the larger the sample size, the more likely the results are to be significant.

The main problem with them making a claim based on a sample size of 50 wasn't the fact that 50 wasn't large enough to be significant, it was the fact that the sample size WASN'T ACTUALLY 50! The sample size would have been in the thousands, maybe hundreds of thousand, because it was measuring home sales, not years!

You write:

"If, as you suggest, house prices were to be analysed by the day, then the sample size for each subset would be drastically smaller, since fewer transactions occur each day."

Huh? I'm not talking about measuring it intraday, I'm talking about measuring it interday!

Reply
Not Unique Name
11/12/2010 06:08:52 am

Anonymous,

The sample size may have been generalized to 50 years to make it more understandable to the general populace. The thing is, even if we had home prices for every single home sold in that time frame, that even though a few had declined, the VAST MAJORITY (upwards of 90-95%) had their value increase. Therefore, NoSuch's claim is logical and you are arguing over a little detail which makes no difference to his claim.

Reply
Anonymous
12/9/2010 02:06:58 pm

You're contradicting yourself. In your initial blog you say...

"So, they were making a claim based on a sample size of only 50. "

And now you say...

"The main problem with them making a claim based on a sample size of 50 wasn't the fact that 50 wasn't large enough to be significant, it was the fact that the sample size WASN'T ACTUALLY 50!"

So, which is it?

It seems that your criticism of the measurement of house price calculation has shifted. You now seem to be saying that individuals should look at daily, rather than annual, aggregates. There are a lot of reasons why this should not be done. These include such factors as seasonal differences. Using an average calculated across a mean helps to even these out.

Reply
12/9/2010 02:07:18 pm

Anonymous,

I did say:

"So, they were making a claim based on a sample size of only 50. That is not even close to being a significant enough of a sample size for making predictions"

I was correct that a sample size of 50, when it comes to something with such high standard deviation as housing prices, is not close to being significant enough of a sample size.

However, it's bizarre that you pick and choose a quote and then claim that I was claiming that sample size was the main issue. I did not.

Before I even made the quote above, I wrote:

"But the people who made decisions based on the 50+ year housing history were actually making decisions based on the sample's number of YEARS, not days, minutes or seconds. This is where they erred. "

Get it? I said THIS IS WHERE THEY ERRED. In other words, that's their MAIN error. They reduced an actual sample size of tens or hundreds of thousand of transactions to 50, whereby if they would've measured the sample size in days, for example, they would've have a sample that was much closer to getting at the truth.

Reply
12/9/2010 02:07:52 pm

Anonymous,

you write:

"It seems that your criticism of the measurement of house price calculation has shifted. You now seem to be saying that individuals should look at daily, rather than annual, aggregates. "

That's just bizarre. When did I claim that people should be looking at annual housing prices? The entire point of my post was that they were WRONG to have have used a sample size of 50 (which meant it was wrong to look at annual prices!)

You mention seasonal differences. Again, that's bizarre. If the sample size includes tens of thousands of transactions, you can't reduce the sample size to 50 annual years simply because you want to avoid seasonal differences.

Reply
Anonymous
12/9/2010 02:10:35 pm

Look, I get that you think you are hot shit when it comes to statistics, but anyone who actually understands the subject will immediately see that you have at best an introductory level understanding of statistics.

Several people have already posted in an attempt to correct you, but you have not understood what they were telling you. Even worse than that, you seem unwilling to admit your mistakes and attempt to learn from them.

By the way, I have a PhD in Mathematics (Thesis in non-commutative quantum field theory) from a well respected university to back this up. Do you?

Reply
12/9/2010 02:10:46 pm

Anonymous,

I know I'm correct. Each person that claimed I was wrong was instead corrected by me.

Instead of making claims about your education, why don't you actually point out where you claim that I'm incorrect?

btw, the average person with a PHD would NOT place 74th out of 1 million+ people on a measure of intelligence. They might typically place between spots 10,000 and 50,000 out of 1M.

Reply
Anonymous
12/9/2010 02:11:25 pm

Unfortunately, you mistake intelligence and knowledge.

While you may be as intelligent as you claim (whatever my opinion on that, I'll accept it as true), you lack knowledge of this subject matter.

The research you critique is a longitudinal research study. It uses the annual average of house sale prices to make observations on the general trends in house prices. Is is more meaningful to look at annual rather than daily averages, because a) the sample size is larger, b) it reduces potential for seasonal bias.

In the analysis of this study, each year represents a subset of data. It is not a unique observation in itself. Therefore, in the study above, the statistical significance of datashifts between years would be calculated with reference to the number of transactions in that year. Please note, that this does not in any way "reduce the sample size".

Please believe me, and the other 'anonymous' above, that you have misunderstood the subject matter. I understand that you see your intelligence as a source of pride and competitiveness, however, no-one is posting with the intention of 'defeating' you, but rather of correcting an erroneous analysis.

Reply
12/9/2010 02:19:20 pm

Anonymous,

you write:

"The research you critique is a longitudinal research study. It uses the annual average of house sale prices to make observations on the general trends in house prices. Is is more meaningful to look at annual rather than daily averages, because a) the sample size is larger, b) it reduces potential for seasonal bias.

In the analysis of this study, each year represents a subset of data. It is not a unique observation in itself. Therefore, in the study above, the statistical significance of datashifts between years would be calculated with reference to the number of transactions in that year. Please note, that this does not in any way "reduce the sample size"."

I completely understand the longitudinal study, and I never wrote anything to suggest that I didn't understand it.

My main argument was simple: that many people claimed that housing prices wouldn't fall because they hadn't fallen in 50 years.

That rationale implies that they believe 50 years to be a large sample size, otherwise why would someone make a prediction based on a small sample size?

Therefore, I challenged the assumption that 50 years is a large sample size. It is not. 50 is a very small sample size.

You may counter and say that there is a huge sample size within the 50 years-thousands of transactions. True. But the thousands of transactions WERE NOT what was referenced by the commentators. If those thousands of transaction WERE what was being referenced by the commentators, they couldn't have said "the housing market has never fallen during several thousand transactions" nor "the market has never fallen a single day over the last 50 years" because it is simply false.

Therefore, I completely understand the original longitudinal study. But if someone is going to make comments that reduce the sample size to only 50, then that is a HUGE mistake. Devastating, in this situation!

Reply
1/7/2011 02:17:24 pm

great post.

Reply
1/16/2011 05:17:16 pm

True. But the thousands of transactions WERE NOT what was referenced by the commentators.

Reply
1/18/2011 11:49:31 am

GUCCI,

I never claimed that the thousands of transactions were referenced by commentators. My whole point is that they reduced the sample size from thousands to about 50!

Reply
Anonymous
1/28/2011 12:19:27 am

But they didn't...

Reply
1/28/2011 12:19:37 am

Anonymous,

they didn't what? Reduce the sample size from thousands to about 50? Of course they did...did you read the thread at all?

Reply
Anonymous
1/28/2011 12:19:59 am

They really didn't though.

I'm afraid you haven't understood this at all.

To provide you with an example:

I ask 5 people what proportion of their salary they give to charity. The next year, I again ask 5 people the same question. Responses are as follows

Person Year 1 Year 2
A 10% 20%
B 10% 20%
C 15% 10%
D 10% 20%
E 10% 20%

For my analysis, therefore, I have two separate samples, each of 5 people. It is legitimate for me to say that average charitable giving has increased from just over 10% in year 1 to just under 20% in year 2. This does not mean that there isn't a single case where charitable giving has fallen, rather that the overall trend is upwards. Nor does conducting this analysis 'reduce' my sample size to 2.

Reply
1/28/2011 12:20:15 am

Anonymous,

your point actually demonstrates the same point I was making.

I understand the point you are making, but economists weren't reporting the data as you did.

Economists were saying that the housing market hadn't fallen in 50+ years, as if that actually meant something. If they were using your data, they would have instead said that charitable giving hadn't decreased in 2 years.

In order for the economists to say that the housing market hasn't fallen, they couldn't use the actual sample of, say, 200,000 transactions, because it wouldn't be true that the market never fell from any of those transactions to the next.

Similarly, with your sample, you can't say that there haven't been any decreases either, among your sample size of 10, because there was the one decrease from 15% to 10%.

But, if you effectively reduce the sample size to 2, and average Year 1 and 2, then you can now say that you haven't had a decrease.

One cannot have it both ways-you cannot have the supposed benefit of being able to say there's been no decrease (using a sample size of 2) when in actuality referring to the larger sample size of 10.

It's the CONCLUSION that's important. The economists were saying that there hadn't been a decrease in 50 years, which can only be true if the sample size was 50.

In your case, you correctly reported the data accurately, but only because you referred to 2 different conclusions based on examining two different definitions of sample size (2 and 10).

Reply
Anonymous
1/28/2011 02:01:00 am

I'm afraid this is where your error lies. As I've previously said, you fundamentally don't understand principle of sample size, or of research analysis.

It would be illogical to compare single house transactions. That is because no two houses are exactly the same. There isn't a single 'price of houses' like you would have a price for commodities. You therefore can't compare single transactions, but instead must take an annual average.

In light of this, it's perfectly reasonable to say that house prices have risen or fallen, because you analyse the market as a whole.

In your response to my comment you state that there are two separate definitions of sample size - 2 & 10. This reveals your mistake. That is simply not true. Rather, there are two separate samples, each of n=5.

If you were to talk about the significance of the findings, therefore, you would do so with reference to the sample size of each subset, not the total number of observations, or the number of subsets of observations.

Reply
1/28/2011 02:01:40 am

Anonymous,

You write:

"I'm afraid this is where your error lies. As I've previously said, you fundamentally don't understand principle of sample size, or of research analysis."

Actually, if you believe that to be the case, you need to show where my error lies, and you can't.

You write:

"It would be illogical to compare single house transactions. That is because no two houses are exactly the same. There isn't a single 'price of houses' like you would have a price for commodities. You therefore can't compare single transactions, but instead must take an annual average."

First of all, someone might WANT to compare single house transactions, depending on what their research purposes were.

But even if one wanted to try to ensure that the houses being compared are relatively equal in structure, you are incorrect again. You don't have to take an annual average. You could take a one DAY or one month average, which would include plenty of different types of homes. In one day alone, there are over 1,000 new homes sold in the USA, and the number of existing home sales is 5 to 10 times greater than that.

You have made serious errors in your logic, and any research paper of yours would be thrown away.

You write:

"In your response to my comment you state that there are two separate definitions of sample size - 2 & 10. This reveals your mistake. That is simply not true. Rather, there are two separate samples, each of n=5."

The total sample is 10. If you use the same criteria all 10 times you asked the question about charity, it adds up to 10. You said that the definition remained the same among them both:

"The next year, I again ask 5 people the same question. "

It's called a longitudinal study.

You've actually just destroyed your own argument! In order for your argument to be valid, you needed to include all 10 samples in your study, because you'd need to refer to all 10. Instead, you just claimed that there were 2 separate samples of 5, implying that the two samples used different criteria and hence shouldn't be comparable!

Reply
Anonymous
2/1/2011 07:40:58 am

Firstly, you say:
"The total sample is 10. If you use the same criteria all 10 times you asked the question about charity, it adds up to 10. You said that the definition remained the same among them both:"

But total sample is irrelevant when calculating significance. That was your reason for criticising the article - that the research 'reduces the sample size', and that it therefore doesn't yield statistically significant findings.

You also say: "You don't have to take an annual average. You could take a one DAY or one month average"

But then, as I've pointed out before, you would be subject to seasonal bias, or could have measurements affected by such things as public holidays. It's well documented that houses achieve nearer their asking prices in summer months. Equally, some houses struggle to sell during school holidays, since their potential buyers are absent.


You also say "First of all, someone might WANT to compare single house transactions, depending on what their research purposes were."

Please can you provide me with one example of when this might be useful. All it would show you is which of two houses is the most valuable.

Reply
2/1/2011 07:41:28 am

Anonymous,

you write:

"But total sample is irrelevant when calculating significance. "

Sample size irrelevant? It's called an ANOVA, an Analysis of Variance!

Of course sample size is relevant...the greater the sample size, the greater the probability that the findings ARE significant and not occurring due to chance! The greater the sample size, the more likely you are entering the long run, achieving the expected values (long term!)

You ask me for an example of when someone might want to compare single house transactions. I am not going to bother with this, because it wasn't central to my argument. After I made that point, I went on to say that one doesn't NEED to look at single home transactions, NOR an annual average. They can look at one DAY or one MONTH worth. (Where's your imagination anyway, do you not agree that someone might WANT to compare single house transactions? This is the world of research we are talking about!)

You argue that a one month or one day average is less ideal than a one year average, because the one month or one day average doesn't cover a large enough time frame to smooth out seasonalities.

There is an easy solution-statisticians can adjust the one day and one month figures to "seasonally adjust" them. This is commonly done on many economic reports.

In fact, I believe (but am not sure) that the monthly existing and new homes reports (from the Commerce Dep't and National Association of Realtors) actually ARE seasonally adjusted!

So, I've beaten your logic. But I will go even further. Even IF it was true that one COULDN'T seasonally adjust the one day or one month results, you must remember that you still need to show that that scenario is worse than the alternate scenario: one that IS seasonally adjusted, but has a terribly low sample size of 50.

By comparison, the one DAY scenario, although it may be seasonally biased (in this hypothetical scenario only), has a much LARGER sample size (over 50 years, it would have 18,250 days or so).

So even IF the one day sample size couldn't be seasonally adjusted, you'd still have to show that that particular shortcoming is WORSE than the shortcoming that the annual average has (a very low sample size of 50)!

Reply
2/1/2011 07:42:36 am

Anonymous,

one more thing. You didn't address every point I made. I'm curious:

You wrote:

"That is simply not true. Rather, there are two separate samples, each of n=5."

Why do you consider there to be two samples of 5, and not one of 10, given that the criteria appears to be the same in both scenarios of n=5? (The same question was asked).

Reply
Anonymous
2/17/2011 04:06:28 pm

"Sample size irrelevant? It's called an ANOVA, an Analysis of Variance!""

That's not what I said. I said that TOTAL sample size was irrelevant. It is the sample size of the particular subgroups you are analysing that matters. That is why I refer to two samples of n=5, since that is what is being looked at.

Incidentally, in your second to last post, as throughout this article, you have again misunderstood what sample size is:
"By comparison, the one DAY scenario, ... has a much LARGER sample size over 50 years, it would have 18,250 days or so."

That statement is incorrect. Whether analysing data by day, month or year, the TOTAL sample size remains the same - the number of housing transactions conducted over the 50 year period of study.

Please look up the term, and how it is calculated.

Reply
2/17/2011 04:06:54 pm

Anonymous,

when I wrote:

"Sample size irrelevant? It's called an ANOVA, an Analysis of Variance!"

why would you think I wasn't referring to total sample size?

Also, you originally wrote:

"But total sample is irrelevant when calculating significance. "

and in your latest post you wrote:

"I said that TOTAL sample size was irrelevant."

Both are false. Total sample size IS relevant: the greater the sample size, the greater the probability that the findings ARE significant and not occurring due to chance! The greater the sample size, the more likely you are entering the long run, achieving the expected values (long term!)

You write:

"Whether analysing data by day, month or year, the TOTAL sample size remains the same - the number of housing transactions conducted over the 50 year period of study."

In the context of the point my original argument made, you are undeniably incorrect. The whole point I made was that people were DEFINING the sample size differently when they defined it as 50 years of average annual prices instead of as several hundred thousand INDIVIDUAL transactions.

Here's proof that I'm correct: If you represent the sample as 50 years of average home prices, you can say that prices never went down. If you represent the sample as several hundred thousand transactions, you cannot make the same claim.

Now, using your logic, you claim that the sample is the same in both scenarios.

It's impossible for them to both be the same, because the fundamental characteristics are different in both scenarios (both definitions cover a 50 year time period; yet one scenario results in a fall in home prices, the other doesn't). How could two of the same things result in different outcomes? Therefore, there is a structural difference in the sample definition.

In fact, if it WAS ok to use your definition of sample size, would scientists even bother (or be able to) research any variables AT ALL?

I think this will be a clearer explanation of my reasoning:

Let's say one scientist says that annual average home prices haven't fallen over 50 years.

Then another scientist comes and says "that isn't true. Among several hundred thousand transactions over those 50 years, the prices have fallen many times".

Who is correct? Well, they've used DIFFERENT sample sizes, because they've defined the variables differently!

The first scientist defined it as a comparison between each of the 50 ANNUAL AVERAGES, whereas the second scientist defined it as a comparison between each of the several hundred thousand INDIVIDUAL transactions.

Those are fundamentally different! In fact it is you that doesn't understand sample size (although, to be fair, I do think that many very intelligent people probably have the same incorrect understanding as do you).

Think of it this way: Some of the source data may be the same among the two scenarios, but the questions the scientists ask (the basis of the study itself) aren't referencing the same source data..

One asks whether AVERAGE ANNUAL home prices have ever fallen over 50 years, and reference annual averages of the data; the other asks whether ANY home prices have ever fallen over 50 years, and references each individual transaction.

Reply
Anonymous
2/17/2011 04:07:25 pm

"Think of it this way: Some of the source data may be the same among the two scenarios, but the questions the scientists ask (the basis of the study itself) aren't referencing the same source data."

How can you say this?? The source data is identical!! Each scientists is looking at a set of data for all housing transactions that occur over a 50 year period. The two scientists won't have different TOTAL sample sizes: They're both looking at the same number of individual observations (which is how sample size is defined).

You mistake the number of subgroups being analysed with the sample size. As I said previously, please look up the definition of sample size, and see how it differs from subgroups.

Reply
2/17/2011 04:07:43 pm

Anonymous,,

As usual, you fail to reply to many points I made. It doesn't seem fair to endlessly allow you to just ignore the points that I've outsmarted you on, only to see you continue on with a last ditch attempt to explain away the one or two points that you DO think you may salvage.

This is especially unfair considering that I've responded to every single point you've made.

The source data is the same, but the sample is not the same. If the sample size WAS the same, you could not ask the same question and get two different answers! Which is what's occured.

There was only ONE main question. The question was: Did housing prices EVER fall over the 50 year period?

When looking at the 50 annual averages, the answer is no.

When looking at the several hundred thousands transactions, the answer is yes.

Now, if the sample size is the same in both scenarios (as you contend), and given that the question asked is the same in both scenarios, then it's IMPOSSIBLE for there to be a different answer.

The only way you can get a different answer is if the questions in the 2 scenarios differ(they don't) or if the sample sizes are different. The sample sizes ARE different.

And about subgroups: The data referenced hasn't been divided into subgroups at all. In both scenarios (individual transactions and annual averages), the entire 50 years is looked at. No subgroups were isolated.

Reply
Anonymous
2/17/2011 04:08:02 pm

This is getting a little painful now. Please note that I'm not trying to beat you, rather to stop you spreading a misinterpretation. But, you seem unable to listen until I've responded to every single point you make. So here we go....

-----
"The source data is the same, but the sample is not the same. If the sample size WAS the same, you could not ask the same question and get two different answers! Which is what's occured."

But that could happen if you analyse the data differently. Suppose you looked at the annual mean vs the annual median of house prices over a 50 year period. In each case, you'd have the same sample size and the same data set. Yet one could show a fall where the other didn't.

-----
"There was only ONE main question. The question was: Did housing prices EVER fall over the 50 year period? When looking at the 50 annual averages, the answer is no. When looking at the several hundred thousands transactions, the answer is yes."

True. But, as I've said previously, this is meaningless. To look at each transaction separately doesn't tell you if housing prices have fallen because each house has a unique value. Houses do not have a single price, as a commodity would. While, as you've previously stated, you could look at a daily figure with a seasonally adjusted mean, this wouldn't be as precise as an annual figure.

-----
"Now, if the sample size is the same in both scenarios (as you contend), and given that the question asked is the same in both scenarios, then it's IMPOSSIBLE for there to be a different answer. The only way you can get a different answer is if the questions in the 2 scenarios differ(they don't) or if the sample sizes are different. The sample sizes ARE different."

This is not impossible, since the analysis is different - as shown above.


"And about subgroups: The data referenced hasn't been divided into subgroups at all. In both scenarios (individual transactions and annual averages), the entire 50 years is looked at. No subgroups were isolated."

The data HAS been divided into subgroups. If you look at annual averages, each year represents a subset of data when n=x housing transactions have occurred.

Reply
2/17/2011 04:08:30 pm

Anonymous,,

you are clearly presenting falsehoods, so there is no use allowing you to continue to discuss.

You write:

"But, you seem unable to listen until I've responded to every single point you make. So here we go..."

You've HARDLY responded to every point I've made! That is a disgusting and false claim to make, easily proven by a cursory review.

Here's one that you didn't respond to:

I asked:

"Why do you consider there to be two samples of 5, and not one of 10, given that the criteria appears to be the same in both scenarios of n=5? (The same question was asked). "

You ignored that in your subsequent response!

Since you've started making false claims about what's actually transpired, I am not in the business of making false claims. You have lost.

Here's a passage:

STARTING with my quote

"The source data is the same, but the sample is not the same. If the sample size WAS the same, you could not ask the same question and get two different answers! Which is what's occured."

But that could happen if you analyse the data differently. Suppose you looked at the annual mean vs the annual median of house prices over a 50 year period. In each case, you'd have the same sample size and the same data set. Yet one could show a fall where the other didn't.

END

So, you directly say that if the sample size WAS the same in the 2 scenarios, you could ask the same question in both and get two different answers if you "ANALYSE" it differently.

That makes no sense at all! That's like saying 2+2 doesn't equal 4!

If you ask the same question in 2 scenarios, it's IMPOSSIBLE for the 2 scenarios to analyse the data differently, because the question, by definition, DEFINES the method of analysis. Example: Q. 1) Did the mean annual average ever fall? Q. 2) Did housing transaction prices ever fall? The QUESTION defines the method of ANALYSIS!

Here's more proof:

You claim that looking at the annual medians and annual means, using the same source data, results in the SAME sample, and that in those cases the means could fall while the median didn't.

But if the means WERE different than the median, of course the sample sizes in the 2 scenarios WOULDN'T be the same. They would look something like this:

Annual Mean sample:

yr 1 = $300,000
yr 2 = $305,000
yr 3 = $310,000
up to yr 50. That is the sample of 50.

Annual Median sample:

yr 1 = $250,000
yr 2 = $255,000
up to yr 50. That is the sample of 50.

The two samples are NOT identical, even though they both used the same source data.

This is absolute proof that I'm correct, and if you continue to deny this, it will be equivalent to saying that you don't believe the sky is blue! (You don't believe that a list of means can be different than a list of medians!)

Reply
2/17/2011 04:08:50 pm

You write:

"To look at each transaction separately doesn't tell you if housing prices have fallen because each house has a unique value. Houses do not have a single price, as a commodity would. "

Of course it tells you if prices have fallen. A house is a house! It's acknowledged that not all home prices will have a single value, whether it's variable or not...that doesn't mean you can't ask "whether home prices have ever fallen".

Why do you imply that EACH home would need to have an identical value in order for one to ask whether the price of ANY homes have fallen?

If you don't define a home narrower than the broad definition of a home, then you are looking at all homes. It's that simple.

You write:

"The data HAS been divided into subgroups. If you look at annual averages, each year represents a subset of data when n=x housing transactions have occurred. "

This is absurd. A subgroup of source data refers to a selection of the source data. If there are 100,000 transactions, a subgroup would be, for example, 50,000 of them.

An annual average is a method of analysis of EITHER the entire source data (100,000) OR a subgroup of that data (50,000), and the RESULT is a sample of data (ie 50 annual averages).

You then compare the two samples of 50 (annual means and annual medians) and analyse THEM to find out the answer to your question.

Reply
Anonymous
2/17/2011 04:09:08 pm

What I've been trying to get across to you throughout this is that if I have n transactions over 50 years, and then analyse, for example, the shifts in the annual means, I don't have a sample of 50 annual means, but have 50 groups of a sample size equal to the number of transactions in that year.

You talk about 50 being too small a base for results to be significant. However, that's not how significance testing works. When testing for significance you look at the samples of the subgroups.

As such, when you talk about 'reducing the sample size to 50' this is meaningless.

Anyway, this is the last post I'm going to make on this. Either you are a genius unique to the world that has an understanding of statistics possessed by no other person in the world, or you don't fully understand the subject matter and are too arrogant/ stubborn to look further into it or admit that you're wrong.

Reply
2/17/2011 04:09:25 pm

Anonymous,

disturbingly, you fail to respond to many points I've made. You must find it disturbing to be outsmarted, I bet it doesn't happen too often.

You are a troublemaker, no doubt about it. But I will still respond to your arguments, to deflate you. You are certainly a sucker for punishment!

You write:

"What I've been trying to get across to you throughout this is that if I have n transactions over 50 years, and then analyse, for example, the shifts in the annual means, I don't have a sample of 50 annual means, but have 50 groups of a sample size equal to the number of transactions in that year."

Why do you think that I don't understand what you're trying to get across? Of course I understand what you're trying to get across; you're simply incorrect to say that the sample size is hundreds of thousands of transactions, because one doesn't need to analyse those transactions in order to answer the question as to whether housing prices have fallen oven 50 years.

In order to answer that question, you need a sample of 50: 50 annual means. It's THOSE data that you need to compare to see whether the price has fallen over the 50 years.

In fact, to answer the question, you don't need to see the hundreds of thousands of transaction AT ALL. In order to answer the question posed above, all someone would need to give you is a sample of 50 annual means. That's it.

So, if you don't need the hundreds of thousands of transactions to answer the question, but you DO need the 50 data points to answer it, why would you say the sample is the hundreds of thousands of transactions and not the 50 data points?

You write:

"You talk about 50 being too small a base for results to be significant. However, that's not how significance testing works. When testing for significance you look at the samples of the subgroups."

I never claimed that 50 was always too small to be significant! I claimed that the sample size of 50 answered a question that wasn't the question that should've been asked:

"I would argue that many people (who were otherwise capable enough) who made decisions based on the 50+ year housing history were actually making decisions based on the sample's number of years, not days, minutes or seconds. This is where they erred. "

Why do you lie?

If you feel that I don't understand the subject matter, then why have you chosen to ignore many flaws of yours I've pointed out, and why have I been able to respond well to every single point of yours?

You are simply an unintelligent fool.

Reply
Anonymous
2/17/2011 09:35:05 pm

"In order to answer that question, you need a sample of 50: 50 annual means. It's THOSE data that you need to compare to see whether the price has fallen over the 50 years. So... why would you say the sample is the hundreds of thousands of transactions and not the 50 data points?"

Wrong. Wrong. Wrong. Those 50 measurements don't exist independent of the underlying data. If I calculate the average height of a group of 30 people, is my sample size 30, or is it 1? It's 30.

You say that I must respond to all of the points you've made in order for my argument to be valid. Aside from the fact that you seem able to spin your arguments in to never-ending and increasingly irrelevant tangents that don't need addressing, the whole purpose of your blog is that people simply need to prove you wrong in one area. Please note, however, that I'm not trying to beat you, but merely to explain to you where you've misunderstood the research you critique.

While I may lack your prowess with regard to facebook quizzes, I do have a fair bit of knowledge on this subject area.

Reply
2/17/2011 09:35:26 pm

Anonymous,

you write:

"Wrong. Wrong. Wrong. Those 50 measurements don't exist independent of the underlying data."

What don't you understand? I've already agreed the source data is the same. How could it not be?

That said, the sample that's used to answer the question is a sample of 50, not several hundred thousand!

Do you not understand that in order to answer the question you need compare the 50 points of data to each other, not the several hundred thousand points of data?!?!

You write:

"If I calculate the average height of a group of 30 people, is my sample size 30, or is it 1? It's 30."

What a bizarre example. That example is not consistent with the example in question, one in which there are 50 means being compared to each other!

You write:

"...the whole purpose of your blog is that people simply need to prove you wrong in one area."

That is one purpose of the site, yes. However, it must be frustrating to be wrong so many times in an attempt to be right just once.

You write:

" Please note, however, that I'm not trying to beat you, but merely to explain to you where you've misunderstood the research you critique."

Well, if you TRULY felt that I've misunderstood the research, then you WOULD respond to all of my critiques in an attempt to correct me!

So you are lying...when you say you want me to understood, that is consistent with the idea that you would WANT to respond. But you directly contradicted that by implying that you didn't bother responding because the purpose it to catch ME in an error.

Which is it?

If you don't respond to all of multiple questions that I've already listed, I may not bother responding to your comments. After all, when the score is something like 50-0, and when the game isn't even fair (because you can't admit when you're wrong), then what's the sense?

Reply
2/17/2011 09:37:07 pm

And why did you change your mind about your 2nd last post being your last post?

Your last post didn't make any substantially new argument (related to the main issue), now did it?

Reply
Anonymous
2/19/2011 04:58:43 pm

You know what, I've ignored most of your petty insults because it's pretty clear you're just an insecure pr*ck. However, that's the second time you've called me a liar, and I have to admit that that gets under my skin.

As per my statement, I wasn't going to post again, until I saw another ridiculous post from you, yet again arguing away from the central point. You seem to think that by simply refusing to accept anyone else's views you're somehow 'winning' the argument. You're not. You're just being a moron. You can say that you're 'winning' 50-0. I don't really care, because I'm not trying to compete with you. Regardless, it's simply not true and you're wrong on the single thing that holds your entire pointless article together; that an analysis of 50 years of housing transactions only constitutes 50 measurements.

I can't explain to you in any new ways that it's impossible to 'reduce' a sample size. In fact, you tacitly admit this by acknowledging that the annual means must come from the same source data (and therefore the same number of unique measurements).

As such, I'll try my hardest to drop this argument. Nonetheless, I think you should muse on one thing. Throughout your blog, you continuously offer comment on the thoughts of experts in various disciplines where you possess no expertise (and regardless of what you say aptitude in probability doesn't make you an expert in all statistical areas). And yet, regardless of your own self-acknowledged amateurism you presume that your partial understanding is somehow more enlightened than that of people who actually know the subject inside out. It's pure arrogance, and it stops you from learning. if a truly wise man knows that he knows nothing, then you most certainly are a fool.

Reply
2/19/2011 04:59:26 pm

Anonymous,

you write:

"As such, I'll try my hardest to drop this argument. "

You certainly have been trying, given your avoidance of several points of mine. It must be tough for a smart person like yourself to accept being outclassed.

As for me calling you a liar...was it not true?
And since it's not true, why would it get under your skin? Does the truth normally get under your skin?

You write:

"You can say that you're 'winning' 50-0. I don't really care, because I'm not trying to compete with you. "

If you're not trying to compete, then what was the purpose of revisiting the arguments we've been debating? Obviously you are lying, you HAVE been competing.

And surely the insecure one is YOU, who has resorted to ommission and lying in order to try to save face. What is it that I've written that suggests I'm insecure? Is it my willingness to respond to every single point you've made? lol you absolute fool!

You write:

"I can't explain to you in any new ways that it's impossible to 'reduce' a sample size."

Why would you want to explain that to me? After all, I never claimed that the samplee size was being altered. I said that different sample sizes were being used.

You seem unable to comprehend that 50 is a smaller sample size than 100,000!!!

You write:

"you're wrong on the single thing that holds your entire pointless article together; that an analysis of 50 years of housing transactions only constitutes 50 measurements."

I never claimed that 50 yrs of housing transactions only constitutes 50 measurements. In fact, 50 yrs worth of measurements could produce millions of different measurement and sample sizes...you could calculate the mean, the median, the mode, looking only at yrs 1 to 10, 1 to 20, 1 to 50, etc.

Anonymous, the ultimate compliment is when someone ignores your argument (as do you) and resorts to baseless name calling (as do you) because it implies you've lost the argument and can't address the issues!

Reply
Anonymous
2/21/2011 04:36:13 am

But where's the name calling coming from? You've called me a liar on numerous occasions. You've called me a fool, stupid, et c.

Incidentally, please can you explain to me how the following two statements sit together:

"They reduced an actual sample size of tens or hundreds of thousand of transactions to 50..."
and
"I never claimed that the samplee size was being altered. I said that different sample sizes were being used."


If I had made those statements, you would be calling me a liar. As it happens, I don't think you're a liar, because that implies intention. I don't think you've deliberately contradicted yourself, but simply tied yourself in to knots.

Reply
2/21/2011 04:36:31 am

Anonymous,

you ask me to explain the coexistence of the following two statements I made:

"They reduced an actual sample size of tens or hundreds of thousand of transactions to 50..."

and

"I never claimed that the samplee size was being altered. I said that different sample sizes were being used."

But I also wrote:

"they reported it over 50 yrs, which in EFFECT lowers the sample size if only because it changes the results: many example of falling prices over 200,000 transactions suddenly becomes 0 example of falling prices when the sample is reduced to 50.

So the problem was that they MISREPORTED the sample size. "

So, when I wrote:

"They REDUCED an actual sample size of tens or hundreds of thousand of transactions to 50..."

it's clear (from my other writings) that I understand that they were reducing it in EFFECT, not LITERALLY. It's impossible to change 100,000 transactions into 50 transactions.

This is also supported by the comment I made in the original article:

"If you measure 50 years in terms of seconds (1,576,800,000) or minutes (26,280,000) or even days (7,300), the sample size is moderate to large. "

This comment specifically references a RANGE of DIFFERENT sample sizes (from moderate to large) depending on whether the sample is 50 or larger. Having different sample sizes means, of course, that you aren't REDUCING the sample size-it changes altogether.

Furthermore, the first comment of mine that you quote was written November 2010. I specifically stated in my rules that those comments weren't eligible for scrutiny, because i wasn't as careful to be 100% precise when writing them.

The example you bring up is a perfect example of why I put those comments off limits. What I wrote in November was TECHNICALLY incorrect, but it is clear that I actually understood what was correct and I was simply trying to simplify my explanation for a reader that continually kept challenging me and seemed unable to understood the basic concept, so I tried to assist him by making a claim that I thought he would understand:

"They (IN EFFECT) reduced an actual sample size of tens or hundreds of thousand of transactions to 50..."

Reply
2/21/2011 04:36:47 am

As for the unpleasant remarks...it was you that started making them, unjustified:

(Keep in mind that you made these comments even though I was responding to every single point of yours, with logic, and I believe you were avoiding several points of mine).

You've chosen to make blanket statements about someone's level of understanding instead of actually providing only LOGIC to support your points.

Jan 28:

"I'm afraid you haven't understood this at all."

"As I've previously said, you fundamentally don't understand principle of sample size, or of research analysis."

Feb 4:

"But, you seem unable to listen until I've responded to every single point you make. "


On Feb 17 you wrote:

"Either you are a genius unique to the world that has an understanding of statistics possessed by no other person in the world, or you don't fully understand the subject matter and are too arrogant/ stubborn to look further into it or admit that you're wrong. "

Again, more unpleasant, unsupported comments made by you, and I called you a fool AFTER those were made.

You refused to respond to the following comment of mine made on Feb 4:

"Annual Mean sample:

yr 1 = $300,000
yr 2 = $305,000
yr 3 = $310,000
up to yr 50. That is the sample of 50.

Annual Median sample:

yr 1 = $250,000
yr 2 = $255,000
up to yr 50. That is the sample of 50.

The two samples are NOT identical, even though they both used the same source data.

This is absolute proof that I'm correct, and if you continue to deny this, it will be equivalent to saying that you don't believe the sky is blue! (You don't believe that a list of means can be different than a list of medians!)"

It is clear that you cannot win this argument-because i've made several decisive points that you simply cannot (or refuse to) answer. (Such as the mean/median example above). The MEAN/MEDIAN EXAMPLE LISTED ABOVE IS ABSOLUTE PROOF THAT THE SOURCE DATA IS NOT THE SAME AS THE SAMPLE SIZE, BECAUSE DIFFERENT SAMPLES OF DATA CAN BE GENERATED FROM THE SAME SOURCE DATA.

Therefore, because you refuse to answer most of the points I make, and because of the decisive points I’ve made, I am ending this discussion and will not allow any further posts of yours to be posted.

Reply
3/15/2011 05:27:04 pm


Ideal is the beacon. Without ideal, there is no secure direction; without direction ,there is no life.

Reply
3/15/2011 05:30:20 pm


Ideal is the beacon. Without ideal, there is no secure direction; without direction ,there is no life.

Reply
3/16/2011 05:02:48 pm

The state or quality of mind or spirit that enables one to face danger, fear, or vicissitudes with self-possession, confidence, and resolution; bravery.

Reply
3/29/2011 08:51:21 pm

That is just a great post, I liked very much to go through it.

Reply
3/29/2011 08:51:39 pm

Well, this is my first visit to your blog! We are a group of volunteers and starting a new initiative in a community in the same niche. Your blog provided us valuable information to work on. You have done a marvelous job!

Reply



Leave a Reply.