=20
*=20
*

=20

=20
=20

=20
=20

=20

=20
*=20
*

=20
*=20
*

=20
**
***
*

**Explanation for the critical appraisal tool for Quasi-Experiment=
al Studies (experimental studies without random allocation)**

**Critical Appraisal Tool for Quasi-Experimental Studies =
(experimental studies without random allocation)**

** **

Answers: Yes, No, Unclear or Not Applicable

**1. ****Is =
it clear in the study what is the =E2=80=98cause=E2=80=99 and what is the =
=E2=80=98effect=E2=80=99 (i.e. there is no confusion about which variable c=
omes first)?**

Ambiguity with regards to the temporal relationship of variables constit= utes a threat to the internal validity of a study exploring causal relation= ships. The =E2=80=98cause=E2=80=99 (the independent variable, that is, the = treatment or intervention of interest) should occur in time before the expl= ored =E2=80=98effect=E2=80=99 (the dependent variable, which is the effect = or outcome of interest). Check if it is clear which variable is manipulated= as a potential cause. Check if it is clear which variable is measured as t= he effect of the potential cause. Is it clear that the =E2=80=98cause=E2=80= =99 was manipulated before the occurrence of the =E2=80=98effect=E2=80=99?<= /p>

**2. Were the participants included in any comparisons similar?**

The differences between participants included in compared groups constit=
ute a threat to the internal validity of a study exploring causal relations=
hips. If there are differences between participants included in compared gr=
oups there is a risk of selection bias. If there are differences between pa=
rticipants included in the compared groups maybe the =E2=80=98effect=E2=80=
=99 cannot be attributed to the potential =E2=80=98cause=E2=80=99, as maybe=
it is plausible that the =E2=80=98effect=E2=80=99 may be explained by the =
differences between participants, that is, by selection bias. Check the cha=
racteristics reported for participants. Are the participants from the compa=
red groups similar with regards to the characteristics that may explain the=
effect even in the absence of the =E2=80=98cause=E2=80=99, for example, &n=
bsp;age, severity of the disease, stage of the disease, co-existing conditi=
ons and so on? *[NOTE: In one single group pre-test/post-test studi=
es where the patients are the same (the same one group) in any pre-post com=
parisons, the answer to this question should be =E2=80=98yes.=E2=80=99]*

**3. Were the participants included in any comparisons receiving s=
imilar treatment/care, other than the exposure or intervention of interest?=
**

In order to attribute the =E2=80=98effect=E2=80=99 to the =E2=80=98cause= =E2=80=99 (the exposure or intervention of interest), assuming that there i= s no selection bias, there should be no other difference between the groups= in terms of treatments or care received, other than the manipulated =E2=80= =98cause=E2=80=99 (the intervention of interest). If there are other exposu= res or treatments occurring in the same time with the =E2=80=98cause=E2=80= =99, other than the intervention of interest, then potentially the =E2=80= =98effect=E2=80=99 cannot be attributed to the intervention of interest, as= it is plausible that the =E2=80=98effect=E2=80=99 may be explained by othe= r exposures or treatments, other than the intervention of interest, occurri= ng in the same time with the intervention of interest. Check the reported e= xposures or interventions received by the compared groups. Are there other = exposures or treatments occurring in the same time with the intervention of= interest? Is it plausible that the =E2=80=98effect=E2=80=99 may be explain= ed by other exposures or treatments occurring in the same time with the int= ervention of interest?

4. **Was there a control group?**

Control groups offer the conditions to explore what would have happened =
with groups exposed to other different treatments, other than to the potent=
ial =E2=80=98cause=E2=80=99 (the intervention of interest). The comparison =
of the treated group (the group exposed to the examined =E2=80=98cause=E2=
=80=99, that is, the group receiving the intervention of interest) with suc=
h other groups strengthens the examination of the causal plausibility. &nbs=
p;The validity of causal inferences is strengthened in studies with at leas=
t one independent control group compared to studies without an independent =
control group. Check if there are independent, separate groups, used as con=
trol groups in the study. *[Note: The control group should be an in=
dependent, separate control group, not the pre-test group in a single group=
pre-test post-test design.]*

**5. Were there multiple measurements of the outcome both pre and =
post the intervention/exposure?**

In order to show that there is a change in the outcome (the =E2=80=98eff= ect=E2=80=99) as a result of the intervention/treatment (the =E2=80=98cause= =E2=80=99) it is necessary to compare the results of measurement before and= after the intervention/treatment. If there is no measurement before the tr= eatment and only measurement after the treatment is available it is not kno= wn if there is a change after the treatment compared to before the treatmen= t. If multiple measurements are collected before the intervention/tre= atment is implemented then it is possible to explore the plausibility of al= ternative explanations other than the proposed =E2=80=98cause=E2=80=99 (the= intervention of interest) for the observed =E2=80=98effect=E2=80=99, such = as the naturally occurring changes in the absence of the =E2=80=98cause=E2= =80=99, and changes of high (or low) scores towards less extreme values eve= n in the absence of the =E2=80=98cause=E2=80=99 (sometimes called regressio= n to the mean). If multiple measurements are collected after the interventi= on/treatment is implemented it is possible to explore the changes of the = =E2=80=98effect=E2=80=99 in time in each group and to compare these changes= across the groups. Check if measurements were collected before the interve= ntion of interest was implemented. Were there multiple pre-test measurement= s? Check if measurements were collected after the intervention of interest = was implemented. Were there multiple post-test measurements?

**6. ****Was follow up complete and if not, were=
differences between groups in terms of their follow up adequately describe=
d and analyzed?**

If there are differences with regards to the loss to follow up between t= he compared groups these differences represent a threat to the internal val= idity of a study exploring causal effects as these differences may provide = a plausible alternative explanation for the observed =E2=80=98effect=E2=80= =99 even in the absence of the =E2=80=98cause=E2=80=99 (the treatment or ex= posure of interest). Check if there were differences with regards to the lo= ss to follow up between the compared groups. If follow up was incomplete (t= hat is, there is incomplete information on all participants), examine the r= eported details about the strategies used in order to address incomplete fo= llow up, such as descriptions of loss to follow up (absolute numbers; propo= rtions; reasons for loss to follow up; patterns of loss to follow up) and i= mpact analyses (the analyses of the impact of loss to follow up on results)= . Was there a description of the incomplete follow up (number of participan= ts and the specific reasons for loss to follow up)? If there are difference= s between groups with regards to the loss to follow up, was there an analys= is of patterns of loss to follow up? If there are differences between the g= roups with regards to the loss to follow up, was there an analysis of the i= mpact of the loss to follow up on the results?

**7. Were the outcomes of participants included in any comparisons=
measured in the same way?**

If the outcome (the =E2=80=98effect=E2=80=99) is not measured in the sam= e way in the compared groups there is a threat to the internal validity of = a study exploring a causal relationship as the differences in outcome measu= rements may be confused with an effect of the treatment or intervention of = interest (the =E2=80=98cause=E2=80=99). Check if the outcomes were measured= in the same way. Same instrument or scale used? Same measurement timing? S= ame measurement procedures and instructions?

**8. Were outcomes measured in a reliable way?**

Unreliability of outcome measurements is one threat that weakens the val=
idity of inferences about the statistical relationship between the =E2=80=
=98cause=E2=80=99 and the =E2=80=98effect=E2=80=99 estimated in a study exp=
loring causal effects. Unreliability of outcome measurements is one of diff=
erent plausible explanations for errors of statistical inference with regar=
ds to the existence and the magnitude of the effect determined by the treat=
ment (=E2=80=98cause=E2=80=99). Check the details about the reliability of =
measurement such as the number of raters, training of raters, the intra-rat=
er reliability, and the inter-raters reliability within the study (not to e=
xternal sources). This question is about the reliability of the measurement=
performed in the study, it is not about the validity of the measurement in=
struments/scales used in the study. *[Note: Two other important thr=
eats that weaken the validity of inferences about the statistical relations=
hip between the =E2=80=98cause=E2=80=99 and the =E2=80=98effect=E2=80=99 ar=
e low statistical power and the violation of the assumptions of statistical=
tests. These other threats are not explored within Question 8, these are e=
xplored within Question 9.]*

**9. Was appropriate statistical analysis used?**

Inappropriate statistical analysis may cause errors of statistical infer= ence with regards to the existence and the magnitude of the effect determin= ed by the treatment (=E2=80=98cause=E2=80=99). Low statistical power and th= e violation of the assumptions of statistical tests are two important threa= ts that weakens the validity of inferences about the statistical relationsh= ip between the =E2=80=98cause=E2=80=99 and the =E2=80=98effect=E2=80=99. Ch= eck the following aspects: if the assumptions of statistical tests were res= pected; if appropriate statistical power analysis was performed; if appropr= iate effect sizes were used; if appropriate statistical procedures or metho= ds were used given the number and type of dependent and independent variabl= es, the number of study groups, the nature of the relationship between the = groups (independent or dependent groups), and the objectives of statistical= analysis (association between variables; prediction; survival analysis etc= .).

=20

=20
=20

=20