Probability vs. Statistics

Probability vs. Statistics
Probability is a branch of pure mathematics–probability questions can be posed and solved using axiomatic reasoning, and therefore there is one correct answer to any probability question.
Statistical questions can be converted to probability questions by the use of probability models. Once we make certain assumptions about the mechanism generating the data, we can answer statistical questions using probability theory. HOWEVER, the proper formulation and checking of these probability models is just as important, or even more important, than the subsequent analysis of the problem using these models.
Probability and statistics are deeply connected because all statistical statements are at bot
tom statements about probability. Despite this the two sometimes feel like very different
subjects. Probability is logically self-contained; there are a few rules and answers all follow
logically from the rules, though computations can be tricky. In statistics we apply probability to draw conclusions from data. This can be messy and usually involves as much art as science.
Probability example
You have a fair coin (equal probability of heads or tails). You will toss it 100 times. What
is the probability of 60 or more heads? There is only one answer (about 0.028444) and we
will learn how to compute it.
Statistics example
You have a coin of unknown provenance. To investigate whether it is fair you toss it 100
times and count the number of heads. Let’s say you count 60 heads. Your job as a statistician is to draw a conclusion (inference) from this data. There are many ways to proceed, both in terms of the form the conclusion takes and the probability computations used to justify the conclusion. In fact, different statisticians might draw different conclusions. Note that in the first example the random process is fully known (probability of heads = .5). The objective is to find the probability of a certain outcome (at least 60 heads) arising from the random process. In the second example, the outcome is known (60 heads) and the objective is to illuminate the unknown random process (the probability of heads).