Politically Incorrect Statistics

Education pays!

2011-10-21T05:10:00.000-07:00

What costs society a quadrillion dollars of lost value? Not educating the poor! This is much bigger than the damage done by the worst nightmares models of any global warming. On par with WW3 in lost value. So it is nice to see it getting summarized in this article. On a personal note, my father was the guy who first did the economics analysis of the Perry Preschool project. Basically the curious fact which seems to keep getting rediscovered is that preschool helps later life outcomes. It doesn't help 2nd grade--but it does help finding a job. My personal view is that it would help the 2nd grade marshmallow test which is one of the best predictors of future success in life. But this experiment hasn't been run to my knowledge.

Whining about Parker's reliability

2009-10-25T05:15:00.000-07:00

Robert Parker has changed wine by adding numbers to ratings. Luckily for me, we both like big wines and so I can buy wines pretty much by the number. Unfortunately, these numbers don't seem as verifiable as one might like. What is their accuracy? Without that, it is hard to think of them as real measurements of anything of importance. Recently Parker tasted some wines he had previously rated. The relationshipe between them is pretty weak. For those that like numbers with their graphs, the p-value is about .5. The lower right point is what Parker likes the best now. But showing the power of the market--I can't seem to find this to buy. Others beat me to it?

What the bell curve should have talked about

2009-04-19T10:04:00.000-07:00

The Bell curve book argued that intelligence is mostly inherited. They when on to make a leap--therefore it is not worth while to educate the stupid since they will always be stupid. But they miss the point. In statistics, we are interested in contrasts. So the question isn't whether a stupid person can be made in to a genius, but instead whether his IQ can be increased enough to get him a better job. It turns out, this is pretty easy to do and fairly cost effective. So once again, going for the controlled experiment helps focus attention on what can be changed, and further provides proof as to whether or not the change is worth it. Go statistics!

More ice in my antarctic please

2009-04-18T19:59:00.000-07:00

This one is for Adi. Not all green house related news is reported on equally.

Statistics of War

2009-01-22T13:22:00.001-08:00

I contend that the distinction between fact and opinion in unwarranted. Facts and opinions are really just varieties of statements, which are testable to varying degrees and "true" in the sense that they are supported by evidence of varying quality. So a statement like "rain yesterday" on the TV news is considered fact because it is 1) obviously testable and 2) reliable in the sense that the weatherman has been doing this for a long time and gets it right nearly every time. He also has no motive to lie, incentives to be accurate and consequences for errors. A thoughtful analysis of facts should consider the 1) supplier 2) testability 3) quality of supporting evidence. Here is an example: It was common place in the media to decry the horrors of Israel's war in Gaza. This is most easily done with a lament about the number of civilian deaths. So lets consider a factual claim made by Ethan Bronner in the NY Times on January 10th.

A tank shell landed outside the home of a family in Jabaliya, northeast of the city, killing eight members of the same family who were sitting outside, hospital officials said, bringing the death toll to more than 820. Nearly half of the dead were reported to be civilians.

Now the "fact" (i.e. Statements) here are two: 1) the death toll (820) and the 2) civilian death toll (approx 400). Let's Anaylze them closely:

Are these statements readily testable?

First, Hamas fighters do not wear uniforms. They fight in highly concentrated civilian areas and they readily employ young adults to provide cover. Now of course, these considerations require verification but there are abundant videos on the web that testify to these statements. Attribution of death is further complicated by cases of"friendly fire" or secondary explosions or a myriad of other inevitable accidents caused by placement of the machinery of war in the middle of a city. So the premise that casualty figures can even be determined accurately is questionable. Now this thesis itself suggests its own testable hypothesis: reported casualty figures should be inconsistent and variable. Indeed, that is exactly the case: on January 6th the NY Times reports that:

The death toll in Gaza reached around 640 on Tuesday, according to Palestinian health officials. The United Nations has estimated that about one-fourth of those killed were civilians, though there have been no reliable and current figures in recent days.

The provide a credible estimate of the intrinsic variance. First, note that on Jan 6th it was reported that out of the 640 dead 160 were civilians. Then on Jan 10th it was reported that out of the 820 dead 410 were civilians. So the reported number of total dead in the 4 days between the two Times articles grew by 180 while the number of civilian deaths (which must of course be lower than the total number of deaths) grew by 250. From this contradiction we can prove that the uncertainty in the casualty statistics is at least 100%. It is interesting, for those who like to dwell on MSM bias that the Times' reporters do not suggest that these numbers are inaccurate, only that they my be out of date.

Who is the supplier of these statements?

The data comes from Hospital officials- presumably Palestinian Arabs. Now hospital officals are certainly not in a position to determine civilians from fighters if the latter are not clearly identifiable. Hospital officials by definition are not on the battlefield and therefore can only communicate what they have been told. So the true "supplier" are Hamas officials and other residents.

What is the extrinsic variability of the data?

The intrinsic accuracy of the casualty statistics are very poor; the problem by its very nature is hard to get right. But there are enormous questions related to extrinsic factors that have nothing to do with the estimation problem directly. We have to answer two questions:

What are the incentives that the actual data suppliers have to communicate honestly?
Do the suppliers have a history of error?

Before answering these questions we have to find a strategy behind Hamas' war against Israel. Why do they fight at all? The battle is not even remotely even; Israel could choose to level the strip, sending Hamas as well as the civilian population to its destruction or exile. Somehow Hamas knows this will not happen. How do they know this? Certainly this is exactly how wars have taken place historically; this is what Hamas would do to Israel if they could; it is what Arab states do to each other; it is how Russia handles the Chechnyian; it is what everybody does in Africa. Hamas knows Israel will not wipe them out because 1) it hasn't yet 2) it expects and counts on the "International Community" to prevent a Western Industrialized state with few allies from exterminating a poor, long suffering, basically defenseless, third world society. So to answer question one: Why would they lie about casualties? Because it is a powerful weapon and it is their only weapon. Finally, to answer question 2: Do the suppliers have a history of error? YES. There are numerous examples of outrageously false Palestinian casualty claims: the Jenin Massacre, Muhammad Al Durah, the Gaza Beach Massacre, Green Helmet Man, Fauxtography. Ordinarily, this kind of gross manipulation should lead to an enormous credibility problem which would undermine their goals. It doesn't. Outside of the natural set of ardent supporters of the State of Israel, Palestinian Arab claims are accepted as true until demonstrated false. The quest to find an acceptable explanation for this is the hardest problem of all.

Lilac Bloom

2008-12-09T12:03:00.000-08:00

Researchers at the Climate Change Research Center at the University of New Hampshire have colloborated with the organization Clean Air-Clean planet to study indicators of climate change in the North East. One of the indicators is the start of the Lilac bloom. The hypothesis is that as climate changes and temperatures rise the Lilacs will bloom earlier and earlier. The data consists of measurements made every years since numerous sites around New England: On average, the bloom has begun 1 day earlier per decade. Now is this result "statistically significant"? To assess this, one would need a model whereby some independence can be asserted. What should NOT be done is exactly what was done: a calculation of the P-value assuming independence. Now in a given year the data are measurements at different spatially local locations: so independence among observations and across years is plain silly. Having overstated their sample size by a factor of 30, the small decrease is now quite statistically significant. They should have measured every lilac and increased the sample size by a factor of 1000!

Death to tall people

2008-09-04T17:28:00.000-07:00

Once again tall people are at a higher risk of dieing. If your tall and haven't died in the past decade, you might recall that there was a study that showed each inch of height cost you one year of life. That is a lot of tree branches to run into in order to kill tall people off at that rate. (On a side note, that same year, left handers were told they would die a decade early.) Well the scientific story behind both of these old results is simply that early in the 1900 people were often short and right handed. Later in the century, more people were left handed and more people were taller. Hence if you died and were tall--you were more likely young. This is called a cohert effect. So the NYT article is quite possibly rediscovering the same old bad science. Without enough of a better link--it is hard to know for sure. But that is where my money is.

Is it really so Simple?

2008-08-27T05:11:00.000-07:00

My previous post identified a discrepancy between the global temp series from 1975 compared to the current temp series: the older data shows a remarkable post war cooling period while the latest versions do not. A simple reason for the difference (suggested in a comment by Zeke Hausfather) is that the Newsweek temperature series from 1975 is Northern Hemisphere. The implication is that the cooling trend was a Northern Hemisphere phenomenon only.

A fundamental and often ignored consideration is that there are an extremely large number of degrees of freedom to play with in these types of analyses. These include methods for handling missing data, population growth, the explosion in the number of temperature sites, technology changes and so on. On the one hand is the argument that the temperature "record" is basically invariant to different approaches to these problems. On the other hand, it may be that the degrees of freedom can easily be manipulated to produce the result you want. As a professional statistician I am predisposed to the latter view. So let's check the temperature series separately by hemisphere:

The middle graphs is the Northern Hemisphere Temperature record. As you can see the post war cooling period that was so dramatic in the NCAR graph published in Newsweek has been flattened out. So here it is again: the trend, which is so dramatic in 1975, is no longer even part of the historical record. How can that be?

Vanishing Temperature Trends

2008-07-29T11:58:00.000-07:00

Back in 1975, when we are at the end of a 30 year period of declining global temperatures, the consensus among the climate scientists was a coming ice age. How they could have come 180 degrees in such a short time frame is another strory. Suffice it to say that forecasting is difficult, especially of the future. I have been trying to figure out how global average temperature data comes to be. I know that there has been attempts to revise the historically record. I think the revisions have been really huge. Consider the graph from Newsweek Magazine, April 28, 1975, page 64:

Now this is a smoothed version but it shows that the peak temperature anomaly of .9 degrees (compared to 1880) occurred in the mid 1940s. By 1970 the anomaly was down to less than .25 degrees.

Fast forward to the present and look at the graph provided by Zeke Hausfather at the Yale Climate Media Forum:

The Red line is the smooth version of the average global temperature. Notice how the huge decline from the mid 1940s to the mid 1970s has......

DISAPPEARED!!!

A fun Causality mistake...

2008-05-20T08:21:00.000-07:00

Lew Rockwell writes

A recent study claimed that labor unions increase the productivity of firms. How did the researchers discern this? They found that unionized companies tend to be larger with more overall output than non-unionized companies

Hard to believe the relationship is causal:

In fact, what we have here is a simple mix up of cause and effect. Bigger companies tend to be more likely to attract a kind of unpreventable unionization than smaller ones. The unions target them, with federal aid. It is no more or less complicated than that. It is for the same reason that developed economies have larger welfare states. The parasites prefer bigger hosts, that's all. We would be making a big mistake to assume that the welfare state causes the developed economy. That would be as much a fallacy as to believe that wearing $2,000 suits causes people to become rich.

Sea Ice Continued

2008-05-04T13:01:00.000-07:00

A friend of mine sent me a pointer to an interesting online article written to clarify some of the technical controversies endemic to climate change debate. I learned the following:

yes; it is acknowledged that some scientists have argued that sea ice in the Antarctic has increased until 2002.
the increase is possibly due to changing wind patterns due to the hole in the Ozone.
less warm air in the interior causes more ice to accumulate in the winter.
in contrast, on the exterior, sea ice is shrinking overall.
By 2006, the overall total is lower.

First, let me emphasize that sea ice on the exterior floats (e.g. the Wilkins Ice Shelf) so overall sea levels are inelastic with respect to changes in exterior ice. Second, the Ozone story is post-hoc interpretation; it is not science. Soft scientific analysis is only impressive when it makes successful and risky predictions. How did climate scientists do on that score? In 2006, it was reported that the total amount of ice was shriking at an alarming rate:

THE first survey of gravity changes caused by the Antarctic ice sheet has confirmed that it is shrinking at an alarming rate.

I turns out that the scientists looked at 2002-2005 data and yes, over that time that the total sea ice (mass; as opposed to area) in the Antarctic was indeed shrinking. Woops! If they had actually bothered to consider past data they would have seen the reality for what it is: changes that are well within the limits of natural variation. In fact, 2005 was an unusual large drop in area (as it can easily be seen from my previous graph) and that should have been noticed. A good analysis would have predicted some regression back to the overall mean in area (all the more so if the Ozone hole theory were taken into account). Three more years later and we are right back on the general increasing trend. Yet, our "climate change" guru has this to say

A more recent study based on satellite measurements of gravity over the entire continent suggests that while the ice sheets in the interior of Antarctica are growing thicker, even more ice is being lost from the peripheries.

Alas; the study was not current by the time this was written. It was already contradicted by current data and now with the 2008 numbers it seems that the point is wholly wrong.

Southern Hemisphere Sea Ice

2008-05-01T11:06:00.000-07:00

On the news page of the National Snow and Ice Data Center (NSIDC) there is an article about the possible collapse of the Wilkins Ice Shelf

Antarctic Ice Shelf Disintegration Underscores a Warming World Satellite imagery reveals that a 13,680 square kilometer (5,282 square mile) ice shelf has begun to collapse in Antarctica.

The article is all very interesting, but I wanted more information since I am not sure why we should care particularly about one ice shelf that is comparitively small. I was surprised to discover that this March has seen the largest increase in Southern Hemisphere sea ice since we started measuring: So which is it? Is global warming causing BOTH the melting of the Wilkins AND the increase in total Southern Hemisphere Ice? How on earth can the NSIDC justify such a misleading headline on their news page? A better headline would be "Antarctic Sea Ice increase undermines Warming World"

Yet Another Causal Confusion

2008-03-27T08:49:00.000-07:00

This one is more subtle. As the baseball season is set to begin, a lot of attention is being paid to Joba Chamberlain's possible roles on the Yankees. He will either serve as the principle set-up man or as a starter. Yahoo sports published an article today by baseball "Economist" Vince Gennaro. Vince did some research:

I recently applied statistical analysis to the more than 70 free-agent signings over the last four months. In doing so, I was able to estimate the going rate for pitchers based on both their role – starter, middle reliever, closer – and their level of performance, measured by win shares from “Hardball Times.

And he discovered, to the surprise of no one, that number one starter get paid more than number 2 starters, that closers make more than set up men, etc.. The problem here is that these values are associations. A pitcher's role is strongly co-linear with his talent. Set up men are cheap because in general they are not that good. It is not entirely clear what it would cost for a lights-out set up man. It could be a lot.

Stupidity Meets Bias

2008-02-05T05:21:00.000-08:00

Reuters has had an especially difficult time with logical complements. A headline from today's (Mon Feb 4, 2008) paper shouts that:

U.S. says no one too young for Guantanamo court

The US courts argued the question: "Are all Juveniles exempt from detention in Guantanomo?." The actual verdict: No. Not all Juveniles are exempt. So why the headline? Apparently Reuters (its reporters and editors) have logically equated "Not ALL" with "Every". Indeed, "Every" may be the antonym for "All" but it is not its logical complement. So what does this have to do with statistics? A common technique for solving problems in probability is to consider an event's complement rather than the event itself. Students readily think that "Not all" is equivalent to "Every" when it fact it is "At least one". Reuters, in its haste to make the US look evil has confused this simple point, succeeding only in making themselves look stupid.

Yet another innumerate journalist...

2008-01-01T18:10:00.000-08:00

Pico Iyer writing for the NYTimes has this to say:

Air travel has never been safer than it is right now — and one of the casualties of 9/11 has been our belief that the affluent parts of the world are safer than the desperate parts. These days, if anything, thanks to terrorist logic, the opposite may be true. Besides, statistics have always shown that we are most in risk within a half-mile of our homes: I, 20 years ago, took my holidays in war-shadowed Nicaragua and El Salvador, Revolutionary Cuba and the Philippines during its revolution, and nothing terrible happened to me till I returned to peaceful, protected Santa Barbara, and a forest fire wiped out my house, with me stuck right beside it.

Reminds me of a story Tom Cover once told. He was pulled over by a cop for rolling through a stop sign. Tom exclaimed that "he was only a block from his house!". The officer lectured "that most accidents occur close to the home". To which Tom replied "that most driving occurs close to the home". The officer got the point but Tom got the ticket.

Who is misinformed?

2007-11-12T14:56:00.001-08:00

I just finished a pleasant little book on the power of statistical analysis in modern business called "Super Crunchers" by Ian Ayres. Of course, he tackles some very politically incorrect topics like gender variation (i.e. Summers) and educational philosophies and the role of numerical analysis in determining practice. But he makes some major errors. Here is one. On page 203 he tackles political polls and the journalists who cover them. At issue is a hypothetical contest pitting Laverne against Shirley (is the Pollster Squiggy?). Polls show that Laverne leads Shirley 51 to 49 percent with a margin of error of 2 percent. Ayres attacks the journalist who declares the contest a "statistical dead heat".

"Balderdash! Laverne is a full Standard Deviation ahead (the margin of error is 2 standard deviations). Crunching these numbers in Excell tells us in a few seconds that there is an 84% chance that Laverne currently leads in the polls. If something does not change, she is your likely winner."

This analysis is horribly confused. First, you need to be Bayesian here to even talk about the "chance that Laverne leads". Then you need a reasonable prior. I am sure one exists, that would make this probability 84% but I dont know it. I do know that Ayres goofed. What he is actually doing is a one sided test of the hypothesis that Laverne=Shirley, by calculating the probability that Laverne will poll at least 51% assuming that her true support is equal to 50%. This is 84%. This is still still not right. What he should be testing is the two sided alternative by calculating the chance that either candidate will poll at least 51% assuming equal actual support. This chance is 68%. Ayres is making the error of ignoring multiplicity. The whole example is wrong.

Summers' Vindication!

2007-04-12T11:48:00.000-07:00

Larry Summers, the beleaguered former president of Harvard, was pilloried for suggesting that scientists examine all possible reasons for sex differences in academia-including genetics. Well, Nicholas Wade's article on sex differences in the NYTimes includes the following paragraph: "Several profound consequences follow from the fact that men have only one copy of the many X-related brain genes and women two. One is that many neurological diseases are more common in men because women are unlikely to suffer mutations in both copies of a gene. Another is that men, as a group, “will have more variable brain phenotypes,” Dr. Arnold writes, because women’s second copy of every gene dampens the effects of mutations that arise in the other. Greater male variance means that although average IQ is identical in men and women, there are fewer average men and more at both extremes. "

Genetics and IQ and all that

2006-10-13T11:45:00.000-07:00

Much ink has been spilled about genes for Intelligence and IQ. A recent article in the New Replublic by Steven Pinker, a Harvard Professor of Pyschology, points out that the appearance of an advantage in intelligent is much easier to establish than its causes. So what exacly is the appearance? Consider Ashkenazic Jews. Pinker writes "though never exceeding 3 percent of the American population, Jews account for 37 percent of the winners of the U.S. National Medal of Science, 25 percent of the American Nobel Prize winners in literature, 40 percent of the American Nobel Prize winners in science and economics, and so on". While the average US Jewish IQ is 8-15 points higher than the average US IQ, this smallish difference in mean translates out into a large difference in the tails- large enough to overcome the population inbalance. Larry Summers in his much chattered about speech on Women in Science, challenged researchers to consider all possible reasons for the underepresentation of women in Math and Science departments. Even the possibility that there is a sex linked genetic advantage. He suggests specifically that the causal factor is the differences in group variances. If two populations are Normally distributed then the group with the higher variance will dominate in the tails. As usual it is easier to establish the appearance of a difference than to establish its causes. Could the difference in variance have a biological basis? That is a legitimate subject of inquiry, which is all that Summers suggested. What was the problem? Dean? Inspired by the statistics, I concocted the following exam question: Suppose for the sake of this problem only, that the average IQ for men is 100 with an SD of 15 and the average IQ for women is 105 with an SD of 10. Both distributions are Normal. Assume that there are equal numbers of men and women in the population. What fraction of men have smaller IQ’s that the median Woman? A person is considered a genius if their IQ is greater than 145. What is the relative proportion of men among the geniuses? Is it too PIC to adiminister?

Global warming in pictures

2006-08-07T05:27:00.000-07:00

Using the wikipedia, this guy put together a site that consists of all the graphs he can find about global warming. Since its a wiki, using the wikimedia software, changes are logged. So it is less fun to be biased in writing since it will just be converted by another reader. Hence this represents the "average" belief of the internet. Probably not a good way to do science, but a pretty cool way to generate graphs about a cause.

Anti-science president of a college

2006-02-23T09:08:00.000-08:00

THe president of Lakehead college argues that we haven't proven that wireless internet doesn't cause cancer, so we shouldn't use it on campus. Let's help him out with some more advances:

Where's the proof that going to class doesn't cause cancer? So, let's cancel classes.
Where's the proof that having exams doesn't cause cancer?
Ok, your turn to add some. :-)

How good is an exit poll?

2006-02-20T11:22:00.000-08:00

January 25th marked the second time that the Palestinians held parliamentary elections: according the BBC - "Exit polls suggest that the ruling Fatah party has won a narrow victory in the first Palestinian parliamentary elections for a decade. The first official exit poll suggests Fatah took more than 46%, compared to 39.5% for Islamic militant group Hamas. The official results may not be announced for several more days" Well official results did come out and Hamas won a landslide! getting 70% of the seats in the new parliament. How did this come about? Well, for about a month I have been saying "look! Exits polls are stupid since people can lie and will lie." Of course, this statement is often an accurate reflection of what is going on. But the numbers here tell a much more interesting story. According to final tallys Fatah won 41% and Hamas 45%. Yes, the numbers to reverse, but not so bizzarely as what the direct comparison between the exit poll percentage of the popular vote and the acutal percentage of awarded seats indicate. It turns out that the subtlety lies in the style of the democracy. What Palestine has is some combination of the American two-party system and the European/Israeli multi-party system. In the two party system a party can just barely win the popular vote and still win a landslide. That's what happened. . Dean can tell you more about why this is actually a very good thing. Dean?

The pulse of the world

2006-02-09T05:32:00.000-08:00

Which wars of our two wars have won the hearts of the public? You do recall we started with a war in Afgan? Well, that war has been a true success. We have like 85% support over there. Not so good in iraq. To see a bunch of these sorts of polls, check out this international meta polling cite: World Public Opinion

Aluminum Foil Helmets

2006-01-31T15:14:00.000-08:00

I thought everyone would appreciate a glimpse into recent cutting-edge research on the effects of aluminum foil helmets on blocking signals from the government. For those without the proper exposure to extremely paranoid people, I should explain that aluminum foil helmets are often used to block the reception and transmission of radio signals between the wearer and some nefarious agency, such as the CIA or NSA. A recent MIT study concluded that aluminum foil helmets are not effective at blocking all frequencies of radio signals, and in fact, can amplify certain government-controlled frequencies. These results are disputed, however, by this online post, which criticizes the MIT study for focusing only on simple radio frequencies instead of the more important psychotronic energy emissions. For those of you who don't know about psychotronic energy, the poster suggests you buy his book, though it might be just as helpful (and certainly cheaper) to just rent the original Ghostbusters movie. The real tragedy of this entire dialogue is that very little statistical discussion is present, and data is not provided for any sort of followup investigation. Considering the importance of this issue, I'm surprised that there hasn't been a greater cry for data availability.

Oh once there was a fisherman

2006-01-04T07:05:00.000-08:00

According to the US Department of Labor, 1 of every 718 fisherman died on the job in 1998. In contrast, 1 out 4613 policemen died on the job in the same year. Not exactly, PIC but interesting nevertheless. Thanks, Mike. A question? I had read (can't remember where) that the really dangerous fishing is up in Alaska and the compensation is really high. Confirmation?

Not Causation...

2006-01-04T06:29:00.000-08:00

Anyone watch Law & Order SUV? During last night's episode, a doctor was forced to admit during cross examination that causality cannot be established without experiment. Statistics makes the big time.

Comments

2006-01-01T08:12:00.000-08:00

I have opened up comments to the public.

The curse of choice

2005-12-23T10:50:00.000-08:00

As any consumer knows, having many choices can be a curse rather than a blessing. John Kay points out that the choice "between Tweedledum and Tweedledee may not matter much to the chooser but it matters a lot to Tweedledum and Tweedledee". Taking this further, the fact that there are many choices keeps the average quality at a higher level and thus a choice made at random can be expected to be of a high quality than when choice is limited.

Not all "cluster" diseases are noise

2005-12-22T13:54:00.000-08:00

I recently got diagnosed with Lyme disease. So my knowledge of it seems to have grown over the past few days. Here is a trivia that I didn't know about: (cut and pasted from the FAQ on Lyme disease:

1.03 Why is the illness called "Lyme disease?"
"Lyme disease is named after a small coastal town in Connecticut called Lyme, where in 1975, a woman named Polly Murray brought to the attention of Yale researchers an unusual cluster of more than 51 cases of mostly pediatric arthritis. In 1977, Dr. Allen Steere and Yale colleagues identified the new clinical entity and named it "Lyme arthritis." In 1979, the name was changed to "Lyme disease," when Steere and colleague Dr. Steven Malawista discovered additional symptoms linked to the disease: problems of neurologic involvement and severe fatigue.
It wasn't until 1982 that the causative agent of the disease was discovered by Dr. Willy Burgdorfer. Burgdorfer published a paper on the infectious agent of Lyme disease, and earned the right to have his name placed on the Lyme disease spirochete now known as Borrelia burgdorferi. Borrelia burgdorferi (Bb) has since been the official taxonomic name of the Lyme disease spirochete. (Information from Forschner-Vanderhoof K., Everything You Need to Know About Lyme Disease)

Inanity continued

2005-12-22T10:08:00.000-08:00

Jonathan Clements writes in his "Getting Going" column that rebalancing should be be done less frequently. It seems that this wrongheaded idea follows entirely from another mistaken idea: that markets can be timed. Momentum strikes again! (cue the soundtrack). Clements even goes so far as to quote William Bernstein (an investment advisor from North Bend, Oregon) who pronounces "that there's significant evidence of momentum in asset class returns". Is this evidence statistically significant? I think not. Clements does make one correct, albeit obvious, point: rebalanceing has tax consequences. duh. Why don't we let the fools be fools and we'll go ahead and invest in equal weighted, frequently rebalanced portfolios (Adi, Mike, Cengiz) or the good old value weighted portfolio (Dean).

Breathtaking Inanity

2005-12-22T10:00:00.000-08:00

Judge Jones in his decision to reject the introduction of Intelligent Design into Dover classrooms hits hard: "the breathtaking inanity of the board's decision is evident when considered against the factual backdrop which has now been fully revealed through this trial". Let's hear it for the Judge. Religions can discourage careful scrunity of its details by labeling those who do it from within as sacreligious and from without as intolerant. Coincidence... or Intelligent Design?

Dancing, sex and evolution

2005-12-22T08:59:00.000-08:00

Now here's a study we should replicate as an in-class demo! This study shows that people who are thought to dance better are more "evolutionary fit." As in more attractive. Ok, how was it actually operationally measured? By symmetry.

So, how about a dance competition on the first day and an evaluation of how many hook ups were made on the last day of class. If that doesn't get us in to the DP, nothing will!

Truth as common belief

2005-12-15T06:57:00.001-08:00

The Wikipedia has been in the news a lot lately. It tries to generate truth by the law-of-large numbers. If enough people all push the articles a little bit in the right direction, it should drift towards truth.

This has caused lots of hand wringing in the circles that typically protect truth from the masses. Nature recently compared the wikipedia to Britannica. Here is a self-referential truth statement from the wikipedia about it wiki.

Looks like a tie!

The Regression Fallacy and Baseball Salaries

2005-12-12T15:31:00.000-08:00

The performance of baseball players is typically measured by accumulated totals of productive batting events. Assuming independence (roughly) among plate appearances implies that recognition is based on estimated success probabilities. Those of us working on the Baseball project know that the variance in those estimates are surprisingly large and typically the same order as the difference between players we consider "good" and those that are ordinary. Thus, the team that buys the star of the hour is almost assuredly overpaying. Is George Steinbrenner listening?

Two studies with possibly interesting ethics

2005-12-07T06:16:00.000-08:00

Two studies came out this morning that I thought worth mentioning. The first on how strife in a relationship can affect healing times. The second is about IQ and genetics. It appears that if a male has one variant, IGF2R, of a growth factor, that male is 20 points lower on IQ.

So how to make normal science controversial? Its just data folks? The IQ is pretty straight forward--no one likes the idea of say tracking kindergardners based on a genetics test. But it seems like there is potential in the other study that I just can't identify.

Freaky Economics revisited

2005-11-28T12:10:00.000-08:00

A salient point in my encounter with Lord's paradox is that multiple regression can easily mislead if the data is not represented properly . It seems that our favorite economist Steve Levitt has been rebutted at least in part because he failed to adjust certain inputs in his regression. The Wall Street Journal reports that two economists from the Federal Reserve recalcuated Levitt's regressions, this time adding a few additional variables to account for variations in crime. They also normalizing arrests to be per capita. The Journal reports that Levitt's finding that abortion reduces crime is greatly exaggerated. Are we suprised? I think the whole thing is a post-hoc fallacy.

While we're on the topic of education statistics

2005-11-28T10:43:00.000-08:00

There is a "paradox" discovered by a guy name Lord, concerning the proper way to analyze pre/post test score data. Generally a comparison of difference scores (i.e. post - pre ) finds no statistically signficant difference between groups, but the same data analyzed using regression shows a statistically significant difference. Lord demonstrated this paradox using weights of college freshman recorded at the beginning and end of the school year. An analysis of the difference in weights for each student reveals that the females and males do not gain weight on average: the school "diet" does not act differently on each sex. On the other hand, a multiple regression analysis of the weight change using starting weight and Sex reveals a statistically significant difference between the sexes. Two reasonable methods, two reasonable but opposite conclusions. On the one hand, the difference scores imply that the "freshman diet" acts no differently for men than women. On the other hand, the regression implies that the women tend to gain less weight than a man at the same starting weight. I have created a simple dataset that shows Lord's Paradox in all its glory.The average value of GAIN is zero for both groups. On the other hand, a multiple regression to predict GAIN using SEX and SEPT weight fits a line for both men and women with a common slope but with a statistically significant difference in intercept. That is, at a given september weight the expected GAIN is less for women than for men. What's the truth? In this example, we randomly generated a TRUE weight for each student with men having a higher mean than women and a higher SD as well. The september weight for both men and women is the true weight plus a random normal (SD around 5 lbs). The June weight has the same distribution. Thus, for each student the pair of weights is a bivariate normal with correlation around .9. So under this model, there is NO DIFFERENCE between men and women with respect to the average weight change. So how do we explain the multiple regression? In the simulation it is clear that the multiple regression uses the wrong variable. A man and woman who weigh the same are not the same- the man is more likely to be under his true weight and a woman is more likely to be above her true weight. Both regress to the mean- but the man regresses upwards and the woman downwards. If we repeat the regression this time using SCALED starting weights- that is we subtract the group mean from every individual- the sex effect disappears! That is, there is no difference in expected weight gain for a man and woman who are at the same weight in september RELATIVE to their group mean (180 for men, 130 for women). Is there a conclusion here? yes- multiple regression can easily lead to terrible error. Is Lord's Paradox not really a paradox at all? Perhaps it's just the Regression Fallacy.

Greenhouse gases increasing--but still no evidence of warming

2005-11-25T19:55:00.000-08:00

One of the motivator for this blog was the battle over greenhouse gases. Well, it appears that the greenhouse worry warts finally have some convincing evidence on their side. ( slashdot on green house gases)

Still to be addressed are

what the implications will be
what the economics choices are
is the benefit of doing something larger than the cost.

This last question is the line that I personally draw in the stand. If the benefit is 100 years away, we need almost a 100 times return before it is a positive value project. There are so many other things that sound more important, that I truly have trouble getting excited about this one.

Even physics has its faith based science

2005-11-25T07:15:00.000-08:00

Science is defined not by where the ideas come from but by the fact that the ideas are testable. In this day in age, where we don't like failing any students, we seem to have encourage tests that everyone can pass. If this attitude gets extended to science it will kill it. What makes science special, is that it is defined by tests that can be flunked. In a word, falsifiable.

So the problem with religion based science is not the fact that religion might be used as a source of ideas, but instead that falsifiablity might be dropped. The primary battle that gets time on the news is intelligent Design. But in physics, there is another theory that equally qualifies: string theory.

An article in slate convincingly makes the argument that string theory is no better than ID. OK, so if we did build a particle colider the size of the milkyway, it could be falsified. But somehow that is a distinction without a difference.

Is theory better than a controlled experiment?

2005-11-25T07:06:00.000-08:00

Hirsch argues in a lovely but disconcerting article that controlled experiments in education are a waste of time. When I started reading this, my laptop was at serious risk of flying out the window. But by the end of the article, I was hooked.

I think we would restate hi main claim as saying, "Know your area of application." He argues that we can learn more about education by thinking hard about research from cognitive science and developing theories.

Journal of Obnoxious Statistics

2005-11-18T12:59:00.000-08:00

I'm not sure how much intersection there is between Politically Incorrect and Obnoxious, but it is worth checking out the recent one-shot production of the "Journal of Obnoxious Statistics": http://www.xs4all.nl/~edithl/jobs.htm I haven't poured through all 101 pages myself, but I may some day.

No such thing as Western medicine

2005-11-14T15:22:00.000-08:00

I read last night, in David McCullagh's excellent biography of John Adams that the technique of smallpox inocculation was brought to America by African slaves. Just an important reminder that the defining characteristic of medicine is testing and that statistics is the ingredient that makes this distinction.

Alternative Medicine

2005-11-14T09:59:00.000-08:00

I glanced at the Inquirer this morning and spotted a headline that suggests that certain vitamins can alleviate pain associated with gout. The dull statistician will teach the experiment: the treatment, controls, placebo, blah, blah, blah and finish with a t-test and a p-value. Important? absolutely. Dull? Devestatingly so. The politically incorrect statistician turns the problem around to address the topic of alternative/Non-Western medicine. Alternative medicine is a HUGE industry supported intellectually by a politically correct ideology that holds that all cultures are created equal. Ergo, all medicinal cultures are equal. Perhaps a starting point for the discussion is Richard Dawkins essay "Snake Oil" which appeared in his collection of essays titled A devils Chaplain. Dawkins' introduces a snappy definition of "alternative medicine": any remedy that has never been tested or has been blocked from testing, or failed when tested. Teach this in class and someone, who swears that her headaches were remedied by the Shaman's balm, will walk out insulted.

Does Statistics have to be boring?

2005-11-11T11:30:00.000-08:00

We had a visit to our department recently by Professor Joel Best. Unfortunately, I missed the lecture because of (yet another!) Jewish holiday, but I did not miss the ensuing talk in the play pen we call the Wharton School Stat department. Joel's essential point is that while statistical literacy is essential and irrefutably important it is nevertheless unloved and disrespected. Futhermore, Joel speculates, that this is unlikely to change since there is no obvious constituency that can take ownership of this problem. I aim to remedy this. To do this, we must confront the problem head on: Statistics is boring. Come on, fellows, come clean. Compare our world to other mathematical disciplines. Not looking good, eh? How about computer science or electrical engineering? Do we measure up? These worlds are sparkling with orginality; with important problems solved with complex and elegant techniques. Yet statistics has a leg up on all these fields because it is not relegated to esoteric domains. Statistics is RELEVANT and not just to farmers measuring crop yield, scientists studying worms or doctors administering treatments. Statistics are at the heart of some of the most interesting intellectual debates of our age. Yet, in our classes, textbooks and conversations, we assiduously avoid these topics. Why? Because as professors of statistics, we argue, it is not our place to teach global warming or sex discrimination. Our place is the t-test and the central limit theorem. So we limit ourselves to the straightforward and the dull. I'm falling asleep just thinking about it.