That’s a Great Question
We are in the midst of summer planning for many schools and districts, layering on top the data coming in from spring testing. In the public eye, there is a reliance on percentages to tell a story. While this is a historical reference point for many based on familiarity, it may not answer the questions educators and leaders really have about their data.
To be strategic about communicating results, let’s start first with the question we are trying to answer. Research questions are a formal way to characterize what we are trying to study or learn from the data set. In other words, what do we want to know about the data as presented? Here are a few examples to consider:
Is there a relationship between attendance and performance in my district?
Did the scale scores improve across two cohorts of students?
Notice a couple of things—these questions require some thought into: 1) what data is available, 2) more analysis beyond a simple percentage calculation, and 3) some cleaning or processing of the data itself. Many times the data comes in and sits (and sits). The formulation of a question, the analysis and processing at least acknowledge the value of the data coming in.
Before you present the data to stakeholders, consider these limitations of percentages:
The data does not most likely represent the same population of students over time (i.e., mobility, student count, specific student populations over time).
The data do not reflect specific initiatives or efforts underway at any point in time (i.e., schools implemented new programs or efforts that could be highlighted somehow).
Students with particular needs may not be represented in the data in order to provide additional support. Again, we focus on 82% of students passing. Are we also discussing the 18% that did not?
My final recommendation when speaking about data sets is to be clear about what is known and not known. Sometimes, we observe a 1% increase and believe that is significant. A percentage alone cannot represent significance. When we present this information, consumers need to understand what we know or do not know. It is appropriate to share, “that’s a great question. We would need to do that analysis to confirm.”
…(and then we do the analysis).