Data Driven Leadership in Absence of Data

Mark E Shanoff, Ed. D
President, Florida Association of School Administrators
Principal, Ocoee Middle School 

Currently, the students, teachers and school leaders in Florida are leading their schools without the standardized assessment data we have become accustomed to in years past. What’s worse is when the data is eventually released, the validity of the data has been called into question by practitioners and leaders from across the State. And, once the data is released in January, more instructional time will have passed since the students took the exam than instructional time exists before the next round of FSA testing.

It used to be that the politics surrounding student assessment, albeit distracting, didn’t impact the planning and delivery of an instructional program based on the previous year’s data. In fact, school leaders relied on that data, to make placement decisions and lineup changes.

But today, the politics of student assessment and the absence of any actionable FSA data now impact the instructional program of Florida’s students, teacher planning, and long range strategic planning by school and district leadership teams.

How does a school or school system operationalize data driven decision making in absence of data? It comes back to the efficacy of your systems within your building:

• capacity building in your leadership team;
• determining what gets monitored;
• how planning will take place
• data to be collected
• how leaders will expect teachers to interact with/act upon the formative data;
• develop a summative solution
• build data literacy as a conative skill in students;
• build automaticity into that system

First, identifying the opportunities for growth among your leadership team. The reason why you appoint folks to your leadership team is based on their impact on a school-wide level, so building their data expertise will give a school leader more mileage out of building a data driven culture than hoarding that expertise in the principal’s chair. Engaging your leadership team in establishing norms for local data-driven decision-making will foster greater buy in from your leadership team and a common expectation among the various areas on your campus each oversees. From a district level, common, ongoing and consistent principal training will assist in helping principals turnkey a district’s common language of data driven instructional leadership. If using data to drive instructional decision making is a district priority, then establishing that common language, the expectation for data analysis and decision tree development based on data creates a uniform standard by which all schools can abide by and replicate, ensuring that in high mobility districts, struggling students or accelerated students will receive consistent access to either intervention or enrichment activities as described in statewide models like the Florida Continuous Improvement Model (FCIM).

As a building or district leader, you must determine what will be monitored. Are you interested in monitoring just the subjects that are assessed via state standardized test and state EOC, is your focus core content areas, or is your focus all subjects, regardless of EOC or state standardized test? Whatever your focus, make sure it’s something you can monitor. My recommendation is to start small, perhaps with a subject area, department or collaborative group of teachers that can help create meaningful assessments, administer the test and analyze the results. Doing so will allow you to build a scalable, sustainable model for data driven decision making, ultimately making it meaningful for teachers, who can make it meaningful for students.

At Ocoee Middle School in Orange County, FL, we have created assessments in our core content classes administered through an online platform called Moodle. Moodle allows teachers to build tests that look like our state test and our Algebra I and Geometry EOCs, which call for students to not only select the correct answer, but also interact and manipulate the information on the screen. This system has taken two years to build, and we have encountered challenges along the way. Ultimately, our teachers want the data more than anything. So, we felt it was a happy medium to simply focus on developing meaningful formative and summative assessments through all core content areas first. We will work on expanding our monitoring structure to include electives in future years.

Data analysis is the culmination of assessment development, administration, scoring, and collection. It requires a backwards design, starting with leadership’s intent by pursuing a data driven strategy for student achievement. Involving teachers in this process of developing intent and working backwards will help build ownership and begin to stamp your school’s data culture.

The term, data rich, is overused in schools. Data rich can mean reliable and valid or refer to the quantity of available data. I prefer to use data rich as a qualitative descriptor, while also describing the school’s or district’s opportunity and structures that could allow for reliable and valid data to be used in an organic, instructionally useful way.

Determining what you will assess, and how you will assess it, is the transformative element of building a strong school culture tied to data. At Ocoee Middle, we not only look at what we assess, but also the question stems, question types, length of assessment, testing environment—online or paper/pencil—who is assessing the student, and how quickly the assessment results can be flipped to help inform instruction at the earliest possible moment.

We create parameters by which we request the teachers review their data on both a micro level and a macro level. We want to see how well students did on a particular assessment versus other students who took the assessment and teachers that gave the same assessment. This is the best way to identify the black belts within a content collaborative. Basketball teams don’t always play their top five shooters. They play their best scorer, their best defender, their best rebounder, and their best passer. Leaders must allow their content collaboratives to develop the same way. One language arts teacher will get better results from their assignment on context clues, while another teacher may be the writing expert for the collaborative. Those roles allow teachers to lean on each other and employ their own microsystems about planning and instruction that isolate the best teachers to plan for and inform the rest of the team about what instructional strategies match a particular benchmark. This process can work at both the formative and summative levels, but must be taught to the teachers, as school leaders will not have the time to meet with each collaborative about each formative assessment. Those meetings should be reserved for summative assessment debriefings.

If summative assessments are the end points of your journey, your formative assessments should be your GPS system. After all, the learning journey is measured by the shorter, sometimes informal checks for understanding more so than the long tests our students aren’t fond of.

Your formative assessments should build into a summative assessment that captures student performance across benchmarks. The summative assessment items should be fully vetted through the formative assessment delivery; meaning teachers should already know whether some question stems and item types have proven to be valid or reliable.

At every level, we pride ourselves in building data literacy as another academic skill for our students. Namely, during our focus groups, we require students to check their overall progress, check their performance on assignments, and understand trends in their own data in each subject. As leaders, we spend time working through our formative assessment data disaggregation with our teachers, but how do we build capacity in our students to monitor their own progress? Students thrive in school cultures that value and celebrate progress through data. To critics who say that students should not be relegated to a number, I say you haven’t leveraged data literacy as a cornerstone of your school culture. Student processing and reflection of one’s own data matters as much as taking any assessment, so provide the time for students to reflect on their progress towards meeting the goals of a particular unit of instruction or class. Data literacy is a necessary conative skill, like organization, time management, and self-advocacy. When we take the time to build data literacy in our students, we provide them with a necessary life skill that they can generalize to personal financial literacy, health and fitness—all pertinent life skills.

Building a data rich culture requires a systems approach and visionary leadership. Leaders must ask themselves an important question—do I value data enough to make it a cornerstone of the collaboration process in my school, combining the expectations set forth by leadership, the talent and expertise of the teachers in teams, and student ownership of results with a focus on personal progress and improvement? If so, it shouldn’t matter whether state standardized test results are available. Because you are creating a data-driven culture based on homegrown assessments, vetted for reliability and validity, in a system that allows time for backwards design—where assessment development and lesson planning occurs simultaneously.

When integrated, lesson planning and assessment development are the 1-2 punch that provide teachers with the checks along the way that inform both practice and progress towards standards mastery.