No Child Left Behind was not entirely an evil piece of legislation. There are good things that came from it. There had been a time in American education when teachers could shut the door to their own classrooms, tune the rest of the world out, and claim, “Well, I taught it. They just didn’t learn it.” That day is gone, and this is a very good thing. Because schools are under such intense scrutiny (not always such a good thing), educators cannot remain in their silos, uncaring about measurements of the outcomes of their actions. No, now we are driven by data. Data comes from evaluation and research, and we need it to support decisions to abandon poor instructional practices, in favor of effective ones. The question for educators now is what kind of date do we need, and how do we get the data that we need. There are three types of research, quantitative, qualitative and mixed method.
Quantitative research is intended to fit a specific focus. It can use larger sample groups than quantitative research can, meaning more people can provide a wider swath of examples. It is used to explain the effectiveness of groups or programs, providing very specific and targeted information about them. It is much easier to interpret data from quantitative than qualitative data. It’s constructivist. A variety of people may be brought in as samples, and it seeks causation between treatments of a group or circumstance and outcomes. It’s transformative. It can examine the experiences of individuals. (Mertons, 2018) Common methods for quantitative research are tests and surveys.
Qualitative research is very different indeed. It is more explorative in nature than quantitative data is. Quantitative research is often constructed with the knowledge of what it’s look for, while qualitative research is more open-ended. Qualitative research is more largely based on reasoning and opinion, and can therefore, sometimes involve bias that makes it harder to interpret than quantitative research. Methods for qualitative research include focus groups, case studies, ethnographic studies, personal interviews, and observations in a particular setting. (Mertons, 2018)
To illustrate the difference between the two, as well as to touch upon the value of the mixed methodology, research being conducted at Media Arts Collaborative Charter School (MACCS) regarding its deployment of project-based learning will be shared. MACCS underwent a middle school redesign in 2015. Embraced with PBL, mixed grad classrooms, team teaching across content, and skills-based grading centered around the acquisition of 21st Century Skills. Now, in its 3rd year of redesign, MACCS has a need to evaluate its success.
A fundamental piece of quantitative date comes from our NWEA (Northwest Education Association) tests. The NWEA is taken by MACCS students three times a year, once in fall, when students return from summer break, next in winter, just before winter break, and finally just before leaving school for summer. It is an adaptive test which provides cut scores representative of student averages in various aspects of reading and math. MACCS measures student progress three times within a year as well as overall progress throughout a student’s middle school career.
In looking at the success of the middle school redesign, MACCS examined the NWEA scores of students who had been at the school for all three years of the new practices. The results both in math and reading were encouraging. By way of a simple example, when MACCS looked at the number of these students who moved from being proficient readers to masterful ones, significant growth occurred. In the category of “literary texts: key ideas and concepts” 6% of the 6th grade group were masters, 17% of that group were masters in 7th grade, and 27% were masters in 8th. In the category of “informational texts: key ideas and concepts” 12% of the 6th grade group achieved mastery, 21% of that group achieved mastery in 7th grade, and 34% achieved mastery by 8th grade. It seems highly suggestive that, based on this data, our middle school redesign is a success. A next step in this area, currently under weigh, is working with NWEA to get data from previous years, when MACCS was using traditional teaching models, for comparison.
Recently, for a second piece of quantitative data, a survey was produced with the support of New Mexico Highland University’s own Dr. Ludmilla Layne, to learn about whether or not MACCS was effectively following the established model for successful project-based learning. The survey had questions about such aspects of PBL as the nature of the problems being solved, group work, opportunities for reflection, real-world connections, etc. Students responded to statements, which were followed by four sub-statements which they ranked from 1-5. 1=strongly disagree, 2=disagree, 3= neither agree nor disagree, 4=agree, and 5=strongly agree. Here is a sample question.
I am clear about the work I need to do on projects because ____.
___ The teachers give clear directions about projects from the start.
___ Teachers support groups by responding clearly to questions.
___ Teachers use clear prompts about the problems needing to be solved.
___ Teachers provide good feedback about how to do the work.
Revealed with this quantitative data, very unambiguously, was that the MACCS Middle School is superb at providing excellent, varied and useful resources to help support students in PBL, but as is the case with most school’s PBL programs, according to literature reviews on the subject, MACCS fails to provide the time and opportunity for students to reflect about their own learning and metacognitive development. Naturally, this is a very limited view of the research at MACCS, but these serve as examples of how quantitative data may be acquired.
MACCS has yet to fully develop a qualitative example of research into its redesign, but now that the school is in its third year of the redesign, the following has been proposed. With our new principal, fully in support of the survey the students took, well aware of the strengths and areas of improvement for our PBL practices, he has planned to observe classrooms based upon our highest to lowest needs. MACCS is discussing and seeking to agree about a format for these observations that will provide for us the most fruitful information. MACCS is also currently developing open-questions for a focus group. Included in this group will be 5 lower performing students, 10 mid-range students, and 5 high performing students. These will all be students who’ve been at MACCS throughout their middle school career. Within each group, the plan is to have 2 students with IEPs participate. Questions being discussed for this flow along the lines of, “I tune out in class when ___”, “What do you feel you’ve learned most about yourself?”, and “What can you do best now that you couldn’t do at the beginning of your time in middle school?”
This qualitative approach is in its early stages of development, but one can see the direction that will be taken. What it finally underscores is that mixed methodology may be the best means by which to really get the big picture of groups and programs. When MACCS has analyzed and evaluated both its quantitative data, in the form of tests and surveys, as well as its qualitative data, via observations and a focus group, it is reasonable to assume that the middle school will have a very good understanding of the success of its redesign.
SOURCES CITED
Mertens, D.M., 2018, Research and Evaluation in Education and Psychology: Integrating Diversity into
Quantitative, Qualitative, and Mixed Methods, SI: Sage Publications
University of Utah School of Nursing, What is Quantitative Research, accessed 2018, 13 September,
www.nursing.utah.edu