Renaissance Teacher Work Samples Consortium

---

TWS at Valdosta State University

Our use of Teacher Work Samples (TWS’s) in teacher preparation programs

The Dewar College of Education uses the Teacher Work Sample as a key assessment to demonstrate candidates’ ability to positively impact P-12 learning in the following teacher preparation programs: Business Education, Early Childhood Education, Middle Grades Education and Secondary Education during clinical practice.

How long VSU has been using TWS’s

Candidates in Dewar College of Education have been developing Teacher Work Samples during student teaching since Fall 1999. The guidelines our candidates use to complete the TWS were revised from the original TWS document developed by the Renaissance Partnership for Improving Teacher Quality. The TWS was adopted because it aligns with the unit’s conceptual framework. All Business Education, Early Childhood Education, Middle Grades Education and Secondary Education candidates are required to complete a Teacher Work Sample as partial fulfillment for clinical practice

How the TWS used at VSU compares to the RTWS

The Teacher Work Sample was modified from the Teacher Work Sample developed by the Renaissance Group and includes the following components: Contextual Factors, Learning Goals and Assessment Plan, Design for Instruction, and Analysis of Student Learning and Reflection. Candidates are taught and practice how to develop the components of the Teacher Work Sample throughout their professional course work prior to student teaching. Candidates plan and implement a minimum 10-day unit based on a specific content area in early childhood education. Mentor teachers and university supervisors work with the candidates to decide on the specific content area. Candidates design instruction to meet individual student’s needs after gathering data about the community, school, classroom, and individual students. After learning goals are developed for the unit, candidates design and administer a pretest to determine students’ knowledge of the subject. The unit is then taught, post test administered and aggregated and disaggregated data analyzed. The candidate then reflects on the instruction based on the analysis of data.

How TWS is used as an evaluation tool

Each candidate enters his/her TWS data into LiveText®, the unit-adopted data collection and analysis system of the Dewar COE. Using LiveText® provides centralized access to all assessments, plans, reports, and evidence of learning. Programs have simple click-ability in reports to view the disaggregated data or granular details of a report in a separate menu. Program faculty members and administrators have access to individual P-12 student data used to determine impact.

How VSU achieves scoring reliability in judging performance on TWS’s

We conduct extensive training with University Supervisors on the Teacher Work Sample rubric. Data are analyzed annually and discrepancies in scoring are evaluated by accessing the assessment and the assignment in LiveText®

How TWS’s have impacted teacher preparation programs at VSU

For additional evidence, the Dewar COE and the departments offering teacher preparation programs have chosen to use effect size indices to describe the magnitude of treatment effects observed (impact on P-12 learning) from the TWSs. Effect size measures are typically used when statements regarding difference in means are desired when they involve studies with different variables, as would be expected from multiple teacher work samples. Additionally, effect size indices provide for a consistent scale for comparison. Generally, and as applied to programs in the education unit at Valdosta State, an effect size of .8 is considered large.

By using effect size indices, the overall impact on P-12 learning for the unit can be described; and by disaggregation of data, effect sizes for individual programs and even individual candidates can be generated. Calculated effect sizes for impact on P-12 learning resulting from intervention provided by Early Childhood Education candidates the past three years are:

	2006-2007 (n=68)	2007-2008 (n=129)	2008-2009 (n=162)
Effect Size Cohen's /d/	1.39	2.02	1.97
Number of P-12 students impacted	1270	2036	3079
Size of effect	Large	Large	Large

From these three years of data, it is evident that ECE candidates at VSU can demonstrate their ability to positively impact P-12 student learning.

Using TWS assessment results and effect size data has allowed program faculty members to have an overall view of our candidates’ impact on P-5 student learning and make program improvements. For example, when the TWS was first implemented, we found the candidates were having difficulty developing pre-post assessments that were aligned to learning goals and that provided the appropriate assessment data on which to base instructional decisions. We decided to embed a similar assignment in the professional semester prior to student teaching to allow candidates more opportunities to practice. We also used TWS data to inform the development of a new series of courses on assessment, planning, and instruction that will be proposed as part of an overall program revision. Inconsistencies in the effect size data prompted us to review individual Teacher Work Samples. We found university supervisors were facilitating the TWS process differently. This led to the development of a training session for university supervisors.

One unexpected outcome for the ECE program has been the interest observed by our candidates as they enter their pre/post assessment data from their Teacher Work Samples into the effect size calculation page provided to them in Microsoft Excel©. Not only are the data available for assessment of program outcomes, but our candidates learn to self-assess the magnitude of change resulting from their instruction. One candidate with a relatively low effect size for her TWS gave the following comments in her reflection: “In my future planning, I will develop lesson plans to include center time to practice and reflect on the number combinations. In the future, I will do an assortment of whole group and small group instruction during each lesson. I will make sure my teaching is differentiated to meet all of the students’ needs. All students are different; therefore, the lessons will be designed to accommodate all learning styles. I will also continue to develop pre/post tests to assess the students before and after a new unit.”

Contact Person

Julia M. Reffel, Professor
Director of Unit and Program Assessment
Dewar College of Education
Valdosta State University
1500 North Patterson Street
Valdosta, Georgia 31698
(229) 259-5132 (office)
(229) 333-7167 (fax)
jreffel@valdosta.edu