When should an observation be regarded as evidence for a proposition?
First, I will make clear what the words "observation", "evidence" and "proposition" mean here.
A "proposition" is a statement about the world, i.e. a statement that corresponds (through the interpretation by language speakers) to some state of affairs in the world, and is true if this state of affairs holds. In this text I am interested only in propositions about the real world.
"Evidence" for a proposition is any thing that increases the estimate of the probability of the truthfulness of the proposition. Note that this definition means that what is evidence is dependent on the agent that estimates the probability. That is why the question in the first sentence is not the simpler question "When is an observation an evidence for a proposition?" The answer to this question is actually agent-dependent, unless it is interpreted as if it is the question in the first sentence.
The word "observation" is derived from the verb "observe", but the meaning in the current context is much wider. The actual definition is something like: "A proposition that it is agreed (by all relevant agent(s)) to be clearly true." This definition is clearly dependent on the agent(s), and even for the same agent, some propositions may change their status over time. In this text, I am interested in those cases where the status of the observation is not disputed.
The definition of observation includes many propositions that cannot be observed in the normal sense of the word. For example, "there are several tens millions people in the UK" is an observation (i.e. a proposition that is clearly true, assuming the relevant agents actually have some idea about the population of the UK) that cannot be actually observed. It should also be noted that the adjective "clearly" is important. For example, the existence of electrons is probably indisputable, but it is not clear that it is clearly true, and therefore it is not clear that the existence of electrons is an observation. As I wrote above, in this case I am interested in propositions on which the relevant agents can agree.
Now the answer to the question in the first sentence: An observation should be regarded as an evidence for a proposition when it is:
This is dependent on being able to evaluate the compatibility of an observation with different states of affairs. Here I assume that it is possible to agree on this evaluation.
An important point to note is that the second part is important as well: an observation that is compatible with other plausible states of affairs should not be taken as an evidence for a proposition.
The logic of this point is actually obvious to everyone, even to people that will find it difficult to make it explicit. For example, assume some people ('the relevant agents') heard John saying that there is a bus in the car park. Should they take this as evidence that there is a bus in the car park? In normal circumstances, they will do it without thinking. However, if there is a doubt, the first question they will ask themselves is: "Would John say that there is a bus in the car park even when there isn't a bus in the car park?" In other words, they evaluate if the observation {John says that there is a bus in the car park} is an evidence for the proposition [there is a bus in the car park] by considering if the observation is compatible with the alternative states of affairs to the proposition (of which there is only one in this case, [there is no bus in the car park]).
Even though the logic is obvious, there is a strong tendency to ignore the second condition above. It is quite common for people to regard as evidence for some proposition an observation which is compatible with many other plausible states of affairs. In some cases, they justify it by considering only restricted set of the other plausible states of affairs (I call this the "Misanalyzing the 'Null Hypothesis'" error in Reasoning errors). In other cases, they don't bother to justify it at all, and seem to do it simply because they like the proposition, and hence accept automatically the observation as evidence (implicitly doing the "conclusion-validation" error, Reasoning errors).
A common way of doing the error of regarding as evidence something that isn't is to do the error in a 'weak sense'. This means replacing the claim that the observation proves the proposition by the claim that it hints or suggest (or similar words) that the proposition is true. As long as the observation is compatible with other states of affairs this is still an error, but many feels that the 'weak sense' is acceptable.
Another significant point of the conditions above is that they are not dependent on the nature of the observation. Any statement, provided it is accepted as observation (i.e. agreed that it is clearly true), is an evidence for any proposition for which it is incompatible with all the plausible alternative states of affairs. An example when this point is important is the conclusion of the Cognitive Brain Imaging paper, which is the observation that there are no reports of replications in Cognitive Brain Imaging. This is incompatible with healthy scientific enterprise, and hence shows that something is rotten in this field and the close fields. This is why the editors and the reviewers oppose its publication so strongly.
=========================================================
=========================================================
Yehouda Harpaz