Listen to the podcast:

 

Shownotes:

Chat

• Research methods course
  
• Public school
  
• Dave and Adam were moody teenagers

 

Articles

I'm going to share my highlighting here with you. This text is taken from Reinhart's website, which is a complete treat to read. But this might help you follow along with the podcast a bit.

Defining p-value

http://www.refsmmat.com/statistics/data-analysis.html

“If we know the distribution of typical cold cases – roughly how many patients tend to have short colds, or long colds, or average colds – we can tell how likely it is for a random sample of cold patients to have cold lengths all shorter than average, or longer than average, or exactly average. By performing a statistical test, we can answer the question ‘If my medication were completely ineffective, what are the chances I’d see data like what I saw?’

“That’s a bit tricky, so read it again.

“If I only test the medication on one person, it’s unsurprising if he has a shorter cold than average – about half of patients have colds shorter than average. If I test the medication on ten million patients…

“The P value is defined as the probability, under the assumption of no effect or no difference (the null hypothesis), of obtaining a result equal to or more extreme than what was actually observed.”

“So if I give my medication to 100 patients and find that their colds are a day shorter on average, the p value of this result is the chance that, if my medication didn’t do anything at all, my 100 patients would randomly have day-shorter colds. Obviously, the p value depends on the size of the effect – colds shorter by four days are less likely than colds shorter by one day – and the number of patients I test the medication on.

“A p value is not a measure of how right you are, or how significant the difference is; it’s a measure of how surprised you should be if there is no actual difference between the groups, but you got data suggesting there is.

“I can get a tiny p value by either measuring a huge effect – “this medicine makes people live four times longer” – or by measuring a tiny effect with great certainty. Statistical significance does not mean your result has any practical significance.

The Base Rate Fallacy

http://www.refsmmat.com/statistics/p-value.html

“Let’s try an example. Suppose I am testing a hundred potential cancer medications. Only ten of these drugs actually work, but I don’t know which; I must perform experiments to find them. In these experiments, I’ll look for \(p<0.05\) gains over a placebo, demonstrating that the drug has a significant benefit.

Go to website to describe the graphic, then come back.

Next time we’ll talk about power

“The chance of any given “working” drug being truly effectual is only 62%[DG1]. If I were to randomly select a drug out of the lot of 100, run it through my tests, and discover a \(p < 0.05\) statistically significant benefit, there is only a 62% chance that the drug is actually effective. In statistical terms, my false discovery rate – the fraction of statistically significant results which are really false positives – is 38%.

“Because the base rate of effective cancer drugs is so low – only 10% of our hundred trial drugs actually work – most of the tested drugs do not work, and we have many opportunities for false positives. If I had the bad fortune of possessing a truckload of completely ineffective medicines, giving a base rate of 0%, there is a 0% chance that any statistically significant result is true. Nevertheless, I will get a \(p < 0.05\) result for 5% of the drugs in the truck.

“You often hear people quoting p values as a sign that error is unlikely. “There’s only a 1 in 10,000 chance this result arose as a statistical fluke,” they say, because they got \(p = 0.0001\). No! This ignores the base rate, and is called the base rate fallacy.

Now look at this cartoon.

http://xkcd.com/882/

 
 

Sound Science

New research shows how you will travel to work in the future!!!

http://www.scientificamerican.com/article.cfm?id=sounds-waves-levitate-and-move-objects

 

 

And finally...

You worked hard to make this far today. Enjoy this:

http://www.wisdomofchopra.com/

 

{plusone}		{ttweet}		{fshare}

 

 

 

{fcomment}

• Prev
• Next