The
Difference Between Normalized Gain g and Effect Size Cohen’s d for
Measuring the Improvement of Student’s Scientific Literacy
As
an undergraduate from Physics Education, I began teaching of Biology at the
secondary school on 22 July 2018 until 30 June 2019 when I acceded to come back
at primary school, both Islamic Madrasah.
Teaching
at the Islamic Madrasah is a hassle because I should consider my perspective on
Islam in teaching. However, teaching at the Islamic Madrasah is not and should
not be considered a burden or chore that just needs to be done. It is a crucial
part of moslem scholar, as we all want to do scientifically sound research and
we should all strive to be effective teachers. Through teaching, we are
responsible for the education of the next generation of islamic peoples, who
will use their own unique ideas and skill sets to advance their society.
Teaching,
in general, should not be seen as a hassle in scholar, but rather as a skill to
be developed and a responsibility to be taken seriously. Teaching does not have
to decrease research productivity, it can greatly enhance research if we allow
it to. One of my evidence about this statement is my experience and work.
After
a year devoted to spruce up the teaching of Biology, I produced a series of
work on scientific literacy related Biology, that continues my undergraduate
thesis, which was related Physics. In these works, I wrote about my experiences
teaching Biology in Islamic Madrasah. Then, I became think to reconsider my
method on measuring student learning. Measuring student learning is a
complicated but necessary task for understanding the student’s improvement and
effectiveness of instruction.
I
have curious about the the difference between normalized gain g and effect size
Cohen’s d for measuring the improvement of student’s scientific
literacy. I used normalized gain g in my undergraduate thesis nor my
first work on Biology Education, then used effect size Cohen’s d on my
latest work on scientific literacy in teaching of Biology. I see need reasons
for using one or both of them, to be explained in any writings on educational research.
So,
in this work I investigate about my curiousity. My investigation focused on the
implications on claims about student learning that result from choosing between
one of two metrics. The metrics are normalized gain g, which is the most
common method used in Physics Education Research (PER), and effect size Cohen’s
d, which is broadly used in Discipline-Based Education Research (DBER)
including Biology Education Research (BER). Data for the analyses came from the
research about scientific literacy on Physics and Biology Education from
courses at institutions across Indonesia.
The
results showed that the two metrics lead to different inferences about student
learning. First, normalized gain g being biased in favor of populations
with higher pretest means. Second, effect size Cohen’s d may mitigate
the limitations of these metric for measuring the learning of high or low
pretest populations of students by accounting for the distribution of tests
scores. Third, by comparing the two metrics across all data, effect size Cohen’s
d is larger than normalized gain g in these cases for the same size
change in the means.
This
work reveals that the bias in normalized gain g can harm efforts to
improve student’s scientific literacy by misrepresenting the efficacy of teaching
practices across populations of students and across institutions. This work,
also, recommends use effect size Cohen’s d for measuring student
learning, based on reliability statistical method for calculating student
learning.
In
addition, using effect size Cohen’s d would allow scholars to use their
work in subsequent studies and meta-analyses, align with the practices of the
larger education research community, nor facilitating more cross-disciplinary
conversations and collaborations as well.
DOI: 10.31237/osf.io/vrwbj
[PDF]