Just an introduction over the main concepts: what are probability and statistics?
Given a random variablewhich can take values in space(the events space), an event is the occurrence of one of the values allowed forand its probability gives the mathematical measure of how likely the event is to occur. It is a number between 0 and 1 (extremes included), where 0 means that the event does not occur at all and 1 that is occurs with certainty.
For example, in the throws of a (fair) dice, the space of events is given by all the faces that the dice can take when thrown (there's 6 of them) and we if we measure the probability of each event we find.
In the frequentist approach, the probability is calculated as the simple ratio of events to the total outcomes the variable can take, that is, as the frequency of the occurrence of event to trials. This assumes a sufficiently large (?) number of trials in the first place and that this frequency will asymptotically converge to the probability of our event when said number of trials goes to.
Also note that this approach inherently entails the concept of repeatability of the process (experiment).
In the Bayesian interpretation, the probability measures a degree of belief. The Bayes' theorem links the degree of belief in a proposition before and after accounting for the evidence, that is, the result of the data observation. In some sense, this interpretation is nearer to the layman's one: the probability encompasses the belief in something, the prior knowledge of the phenomenon at hand.
An example illustrating the difference in the two approaches, carried out using a coin flip, can be found in this blog. A really good read.
Statistics is that branch of Mathematics dealing with the analysis of data, the testing of the reliability of experimental results and the building of models which can describe patterns and trends in the observations.
Descriptive Statistics describes the main features of a collection of data quantitatively, that is, describes a sample without learning anything about the underlying population. It does not make use of probability theory.
Inferential Statistics learns from a sample of data in order to infer about the population.
The probability expresses the fraction of the successes over the total (we are using a frequentist interpretation) and is a number between 0 and 1. The odds of something quantify the fraction of successes to the failures instead and is a concept mostly used in the context of gambling.
If you have possible events , the probability of one of them occurring (the bars indicate the cardinality of the set of occurrences of the event) can be written as
while the odds in favour are defined as
that is, as the fraction of occurrences of event to all occurrences of all other events. The odds against are defined as the reciprocal:
In most cases, odds are reported in notation as (successes : failures) rather than as a ratio, or, very often as where is the probability of success, or as .
In a (fair) coin flip, the odds in favour of a head are 1:1, where the notation uses the third way outlined above.