The value of using data
Ever feel like you are drowning in information and data? Recently I was at a medical appointment with a friend. The specialist was talking about specialized scans and other evaluation tools. He commented about situations where tests provide a lot of information (data) but it’s not always useful. Sometimes there is a lot of information but it does not lead to an intervention. Well, you say, that’s medicine; we are in education.
Wait, I say, educators and school systems collect a massive amount of information (data) for various reasons. But how do we use these data. Which data for what purpose is so very important?
Often, just hearing the term “data” conjures up some unpleasant memories. We’ve all had experiences where it is really challenging to get data that are useful and meaningful for the task at hand. Sometimes the data we have are too narrow and don’t match or provide information we can use for decision-making and planning. Other times, the data may be too broad such as state or district-wide data. We could waste a lot of time trying to get what we need and go into data overload. Rob Horner, Co-Director of the National PBIS Center, always says that in SWPBIS teams need the right data, at the right time.
These reasons prompted the National Center for PBIS to develop very clear, user friendly data systems that are designed for our purposes and accessible to all schools. Many of these data systems are free and there are great resources for acquiring them and learning how to use them well.
So what is the purpose of using data?
In our efforts to implement SW-PBIS, data tools and systems make it possible for us to reliably measure our progress, effectively solve problems and challenges and to measure the important outcomes we achieve.
SWPBIS implementation is a process that typically takes 3-5 years to fully achieve. It is essential that we measure progress along the way. There are several core features that define Tiers 1, 2 and 3. The available tools make it easy for school teams to know where they are in implementation. For new schools, these tools measure the core features we are establishing so we know when we have met out goals for the different tiers. For existing schools, these same tools make it possible to recognize if your school is maintaining core features or possibly drifting off-course. In any case, regular progress monitoring with specific tools designed to be reliable and valid for our action plans.
Second, our school teams work by doing on-going action plans aimed at implementation and student outcomes. Action plans are based on regular data and feedback on our accomplishments. In the initial stages of implementation, your school team might use one of these tools several times during year 1. Ready access allows your team to make changes in real time as needed to achieve your goals. For example, you may learn that some aspects of PBIS are in place but not across the whole school or all staff and students. This specificity allows the team to make very clear changes.
Third, data systems make it possible for our school teams to measure outcomes. The important outcomes we seek are school environments that are predictable and safe and lead to student outcomes that include decreases in office discipline referrals and increases in student achievement, attendance and perceptions of school safety.
Data and evaluation
One of the best aspects of SW-PBIS is the comprehensive, user-friendly data and evaluation system available to schools. Many of the tools in this evaluation system are free (see pbis.apps.org). This system includes tools for measuring:
- Implementation and fidelity
- Stage of implementation (forward and backward)
- Outcomes of full implementation
Evaluation is an essential component for any legitimate program and model. Yet sometimes the idea of doing evaluation can be intimidating. In PBIS, you can sum the evaluation process pretty simply with two questions:
- Did we implement SW-PBIS to fidelity?
- If so, did it make a positive difference in student behavior?
I want to tell you about the tools, where to find them and how to use them. These tools and questions will apply whether your school is new at SW-PBIS or has been doing it for years.
Pulling it all together: Asking the important questions.
All of the tools described in this section are free and located at www.pbisapps.org in the section under www.pbisassessment.org
What questions are you asking? Ask first, then use your data to answer the questions.
Are you implementing PBIS?
Think of your school’s implementation as having a beginning, middle and an end point and that movement is in both directions. In the beginning, you have a PBIS team and the focus is on putting in place the core features of SWPBIS. Some schools may have begun using a tool called the Team Implementation Checklist (TIC). The TIC is a short checklist completed by the school’s PBIS Leadership team. It provides a snapshot of where the team perceives the school to be in the implementation process. It provides baseline data because only a few features may be in place or partially in place. This tool is an easy and efficient way to measure progress in implementing Tier 1. It is typically repeated quarterly each year. The data provide the basis for creating and updating an action plan and measuring progress. It can be useful after full implementation to gauge fidelity, or the degree to which your school is maintaining the key features of PBIS.
A newer tool, that has added value over the TIC and some other tools, is the Tiered Fidelity Inventory (TFI). See https://www.pbisapps.org/Applications/Pages/PBIS-Assessment-Surveys.aspx#tfi The TFI measures implementation of all three Tiers. It is quick, efficient and easy to use. You could use this during the baseline or beginning period instead of the TIC and other tools, and reassess quarterly over a few years. It will give you good data on Tier 1 as well as measuring progress on Tier 2 and 3 without having to add extra tools to your process. The TFI assesses broad components of PBIS at each tier. For example Tiers 1 and 2 include 1) teams, 2) implementation, and 3) evaluation while Tier 3 includes 1) teams, 2) resources, 3) support plans, and 4) evaluation. The goal is to achieve a score of 70% and above on the core features. When schools achieve 70% or more on three consecutive TFI administrations, the team can begin using it as an annual tool measuring fidelity.
The TFI can be used for formative evaluation, on-going progress monitoring and action planning, annual assessment of accomplishments and in some places, used for state recognition programs.
Does PBIS implementation make a difference for students?
Generally speaking, schools won’t see big changes in student behavior until they have implemented PBIS with fidelity. There are several outcome measures schools can use to answer the question about outcomes related to PBIS implementation. These include:
Social behavior (Office Discipline Referral-ODRs rates, patterns). The School-wide Information System (SWIS) is an easy tool for monitoring student behavior. See swis.org
- Academic gains (Math, Reading and Writing)
- School Climate (perceptions of students, faculty, staff, families)
Data are our friend. We must use them. To use data effectively we need valid, reliable and targeted tools that are readily accessible and easy to use. Fortunately in PBIS, we have these. See www.pbisapps.org for more specific information and steps to enrolling.
Using objective measures like the TFI and SWIS help us avoid seeing attractive artifacts, like t-shirts, posters, banners, tickets, prizes and parties as markers of full implementation rather than clear changes in adult and student behavior.by