Statistics Quiz#1

[Ether]

OK, here's an off-season challenge for the stats gurus here on CD:

A computer equipped with a quantum noise random number generator is programmed to perform the following experiment: It flips a biased coin 2000 times and records the number of heads for that experiment. The coin is biased so that it is twice as likely to come up tails than heads for any given flip.

If the computer were to perform 100 of these experiments, what is the probability that more than 59 of those 100 experiments would each have fewer than 672 heads?

[DonRotolo]

You're retired, right? Otherwise you'd never have the time to come up with such devious (and interesting) problems.

Anyway, you can also look at the problem as having a 3 sided coin, and we're only interested in side B, not sides A or C.

I got a C in Statistics. 34 years ago. :rolleyes:

[bduddy]

P(B(2000,1/3)<672) = .5929
P(B(100,.5929)>59) = .4855

Is this a rounding error of 1/2?

[Ether]

P(B(2000,1/3)<672) = .5929
P(B(100,.5929)>59) = .4855


Your approach looks correct, but your result for P(B(2000,1/3)<672) looks a bit off. What app did you use to compute the value .5929?

[bduddy]

Your approach looks correct, but your result for P(B(2000,1/3)<672) looks a bit off. What app did you use to compute the value .5929?


Some web site, I can't remember. Trying another one gets me:

P(B(2000,1/3)<672) = .589957
P(B(100,.589957)>59) = .4616

No?

[Ether]

Some web site, I can't remember. Trying another one gets me:

P(B(2000,1/3)<672) = .589957
P(B(100,.589957)>59) = .4616

No?

Still no joy.

As in your first post, the second computation is correct, but the first is off a bit.

I suspect the web sites you are using implement algorithms which are not accurate for larger numbers.

But hey, you understood how to solve the problem. Reps to you!


PS:

I ran the numbers in Python, Octave, Scilab, and Maxima and they all agreed with each other out to ~12 sigfigs.

If anyone reading has access to Matlab and/or R, could you please post the result for:

p1 = binocdf(671,2000,1/3)

[Ether]

You're retired, right?

Yeah, retired. I highly recommend it :)

Otherwise you'd never have the time to come up with such devious (and interesting) problems.

I recently helped a friend through a tough stats course for his EdD. It got me interested in reading a couple of the neglected stats books in my library :-)

Anyway, you can also look at the problem as having a 3 sided coin, and we're only interested in side B, not sides A or C.

Sure.

I got a C in Statistics. 34 years ago. :rolleyes:

I never took a stats course. Got interested in it later in life.

[Richard Wallace]

Yeah, retired. I highly recommend it :)
...
I never took a stats course. Got interested in it later in life.I hope all you young whippersnappers are paying attention out there.

Follow Ether's lead on this: concentrate on the physics while you are young, and save the statistics for your later career (or retirement). Too many engineers today rely on statistical inference to make decisions, when a relatively small dose of physics would have provided the required insight much more directly.

[Michael Hill]

I hope all you young whippersnappers are paying attention out there.

Follow Ether's lead on this: concentrate on the physics while you are young, and save the statistics for your later career (or retirement). Too many engineers today rely on statistical inference to make decisions, when a relatively small dose of physics would have provided the required insight much more directly.

Taking this approach served me well until I got to quantum mechanics, then it went really downhill that semester

[s_forbes]

Too many engineers today rely on statistical inference to make decisions, when a relatively small dose of physics would have provided the required insight much more directly.

Speaking as a process engineer, this is absurdly accurate.

Also, I feel bad that I ignored the mathematical analysis and my answer to the original question is "a tad less than 50%".

[Tristan Lall]

Follow Ether's lead on this: concentrate on the physics while you are young, and save the statistics for your later career (or retirement). Too many engineers today rely on statistical inference to make decisions, when a relatively small dose of physics would have provided the required insight much more directly.
I have a feeling there's a story behind this. Anything you can share?

(I'd say statistics are extremely useful in quality management for high-volume production, in processes that have already been well-characterized and refined. But that's probably not the scenario being criticized.)

[Michael Hill]

I have a feeling there's a story behind this. Anything you can share?

(I'd say statistics are extremely useful in quality management for high-volume production, in processes that have already been well-characterized and refined. But that's probably not the scenario being criticized.)

For that, I'd just let your company pay for all the six-sigma (lol) training, which would cover everything you need to know.

[Richard Wallace]

I have a feeling there's a story behind this. Anything you can share?

(I'd say statistics are extremely useful in quality management for high-volume production, in processes that have already been well-characterized and refined. But that's probably not the scenario being criticized.)You're right, Tristan -- there are many engineering situations that call for statistical inference; e.g.,

-- quantum models for very small scale material properties - as Michael Hill pointed out above, this is a special case for which statistics is an indispensible part of physics,

-- developing strategies for playing games of chance,

-- detection and ranging methods to extract useful signals in the presence of background noise, and

-- optimizing large scale serial manufacturing processes by isolating defect causes that arise due to variation of many potentially interacting factors.

In each of these situations, the challenging aspect is random variation in the observed data.

However, there are many more engineering situations in which statistical inference is not the best tool. The scenario I was criticizing above arises when an engineer tries to use statistical inference to understand patterns in observed data that are not random at all, but instead are based on physical principles the the engineer has failed to recognize -- in such situations the challenge is not random variation, it is ignorance.

Ignorance cannot be overcome by statistical inference.

[RyanCahoon]

If anyone reading has access to Matlab and/or R, could you please post the result for:

p1 = binocdf(671,2000,1/3)

Matlab gives
p1 =

0.5916

or with format long
p1 =

0.591634715653171

[Ether]

Matlab gives
p1 = 0.591634715653171

Thanks Ryan.

If anybody has access to R, Mathematica, MathCAD, or Maple, would you please post the value computed for p1?

I'm investigating which numerical package gives the most accurate answer.

[Basel A]

Thanks Ryan.

If anybody has access to R, Mathematica, MathCAD, or Maple, would you please post the value computed for p1?

I'm investigating which numerical package gives the most accurate answer.




Maple 17 and Mathematica 9.0 gave the same answer, a very large fraction that evaluates to 0.591634715653168147118256848279....

Here's the code:

Maple (with Digits set to 30):
evalf(CDF(RandomVariable(Binomial(2000, 1/3)), 671));


Mathematica:
N[CDF[BinomialDistribution[2000, 1/3], 671], 30]

[Ether]

Maple 17 and Mathematica 9.0 gave the same answer, a very large fraction that evaluates to 0.591634715653168147118256848279....

Thank you.

That confirms the 80-digit arbitrary-precision (http://en.wikipedia.org/wiki/Arbitrary-precision_arithmetic) calc I did with Maxima.

The "very large fraction" has 953 digits in the numerator (and denominator). The first 80 digits of the decimal representation of that fraction are:

load(distrib)$
fpprec:80$
bfloat(cdf_binomial(671,2000,1/3));

0.591634715653168147118256848279300032681473129301 67045093849129098685265637080124

Comparing the above to double-precision calculations:

0.591634715653168.....(Scilab)
0.59163471565317......(Maxima)
0.591634715653171.....(Matlab)
0.591634715653066.....(Octave)
0.59163471565245895...(Python)

... Scilab is the most accurate, Python the least, and Matlab is in the middle of the pack.

If you do the calcs using double-precision in Maple and Mathematica, what result do you get?