The Threats To Internal And External Validity

Internal and external validity are concepts that oscillate around the result of a study and whether or not it is applicable in all situations outside the area of the study. Internal validity is the extent to which the cause and effect relationship between treatment and outcome is established. External validity determines how applicable the result is to all events. For example, the assurance that the treatment administered to a COVID-19 patient is the cause of the change in the observed outcome.

There is a list of potential threats to internal and external validity that should be addressed when planning a study. These threats pose alternate explanations for the causal relationship between the dependent and independent variables of a test if they are not adequately controlled.

Threats to Internal Validity

History: Events such as natural disasters, political change, etc. affect the participant’s attitude which makes it impossible to determine whether any change in outcome is due to the treatment given or the historical event.

Maturation: It capitalizes on time as an important factor in the course of the study. The experiment naturally takes place over a period of time. Whether days, months or in some cases years during which a number of changes could happen to the participants (loss of interest, mood swing, inattention, old age, tiredness, etc.) when that happens, then it will be impossible to know the cause of the change in outcome.

Testing: This threat is due to repeated administration of the same testing measures. If participants get the same test over and over again, the participants will become familiar with the test and thereby have again in the test score. When there is a large difference between the pre-testing and the post-testing period, test threat is likely not to be a threat to internal validity.

Instrumentation: Precision inaccuracy, measurement error, bad calibration, changing of the instrument used over a period of time, and instrumental delay pose a threat to the internal validity of the experiment that is being carried out.

Experimental bias: This occurs when the groups under observation are treated differently. The discrepancy in the treatment of the groups leads to different outcomes. In some cases, experimental bias may be unintentional and it may be due to honest mistakes on the part of the observer by manipulating the result to match a preferred result. Experimental bias can be avoided by using blind techniques. A blind technique is when the experimenter and the observed subject are blind to the possible outcome of the study.

Attrition: When data is collected differently, people tend to drop out of the experiment due to many reasons such as loss of motivation to go on with the experiment, illness, loss of participant, mobility, unsatisfactory treatment and failure to follow the participant up. When a participant drops out of a study, they will likely discontinue the intervention measure.

The differences between those that discontinue and those that continue will introduce bias into the result. Getting result data for those that pull out of the study will be difficult. For instance, in the study of the response of the immune system to a particular intervention to prevent the participant from having influenza, the result for those that dropped out will not tally those that stayed.

Diffusion or social interaction threat: This is the spread of the study method or treatment from the control group to the treatment group through interaction or when the key people are aware of their existence and the role they play in the study. This threat can be avoided by isolating the groups from each other and being discrete about the data of the people involved.

Threats to External Validity

Selection partiality: As the saying goes “Even identical twins are not truly identical” no two people are the same. Individual differences such as gender, age, intelligence, and so on make the result inapplicable to the general population. This can be minimized by putting a specific criterion into consideration and selecting the samples accordingly. That way, the result of the research will be practicable for all situations that correlate with the subject matter.

Pretest-protest effect: This involves getting the pretext result for the subject of interest before administering a treatment followed post-test after the treatment. If the posttest result for the control group and treatment group is not in concordance, then the application of the result will be impossible. 

Malfeasance: Fabrication of data to support the observer hypothesis by disregarding data that has no agreement with the hypothesis. The result cannot be generalized to areas that match the specific test due to the manipulation nor will the result be consistent with that of another observer of the same subject.

Situational factors: Factors like changes in time, location, climate, environment, and health condition can affect the external validity of a result.

Laboratory and world research: Sometimes, some experiments carried out in the laboratory may not be consistent with the ones carried out in the real world thereby making the lab result inapplicable in the real world.

Time measurement and treatment effect: For treatment that takes a while to yield the outcome, the effect of the treatment may not be known before the required time. In this situation testing immediately after treatment will show no impart until it reaches the required period.

Inexplicit description of the experimental procedure: If the researcher fails to give a passable description of the experimental measures or made a mistake in describing how the study was conducted, it will be difficult to prove that the result can be generalized.

Internal validity and external validity are two sides of the same coin. That is, if a test is internally invalid, the same will go to external validity. The investigator carrying out and reporting the research outcome can address the above threat so that the effect of the treatment will be accurately different and the application of the result will be general. Also, the researcher should be mindful of the factors that influence all the aspects of the research.