|
||||||||||||||||||||||||
|
Middle Tennessee State University
Southeast Missouri State University
|
|
|||||||||||||||||||||||
Renaissance Partnership Members
The Teacher Work Sample (TWS) at Emporia StateEmporia State requires a TWS from ALL candidates in the teacher education program. Spring 2003 was the third semester for ALL candidates to complete the TWS requirement. ESU has collected and scored TWS data for 5 semesters. All elementary and secondary students in methods classes and in Professional Development Schools develop abbreviated versions of the TWS during their junior year. This is an effort to promote classroom connections to the TWS and familiarize candidates with the performance and evaluation process. Both elementary and secondary faculty members have taken a central role in integrating components of the TWS in their courses (See Research and Accountability section below). Although ESU has worked closely with the TWS Renaissance Group as a part of the consortium, ESU’s prompt and rubric is not the same evaluation tool used by the rest of the Group. Variations between the two versions are minor but significant.
During each semester, Dr. William Samuelson and Dr. Anthony Ambrosio hold a workshop with all of the Liberal Arts and Sciences and Business faculty who teach departmental methods courses or visit student teachers (Mentors). The purpose of the workshop is to provide feedback about the performance of students in their teaching majors on the TWS. Data is provided to compare their major students’ performance with that of all ESU students, elementary students, and all secondary students. The data and score sheets are used for feedback to individual University Supervisors.
Fall, 2002
The
Teacher Work Sample (TWS) at Emporia State The TWS is a requirement for ALL candidates in the teacher education program.
During the Fall Semester 2002 all elementary and secondary student
teachers developed a TWS as a part of the requirements for student teaching.
Fall was the second semester for all students to complete the TWS
requirement. ESU has scored TWS
from students for three semesters. All elementary and secondary students in methods classes and in Professional
Development Schools developed abbreviated versions of the TWS to promote
classroom connections to the TWS and familiarize candidates with the performance
and evaluation process. Both
elementary and secondary faculty members have taken a central role in
integrating components of the TWS in their courses (See Research and
Accountability section below). Although ESU has worked closely with the TWS Renaissance Group as a part of the consortium, ESU’s prompt and rubric is not the same evaluation tool used by the rest of the Group. Variations between the two versions are minor but significant. Refinements were made this semester to the ESU Rubric and prompt for factor five. The Glossary of Terms was also expanded based on recommendations from raters at the Spring scoring session. The revisions did not impact weighting or scoring of the student TWS. To see ESU’s prompt and rubric,
the readers should please go to: http://www.emporia.edu/teach/dean/twsm.pdf
Mentoring
Program Report During the fall semester, Dr. William
Samuelson and Dr. Tony Ambrosio held a conference and workshop with all of the
Liberal Arts and Sciences and Business faculty who teach departmental methods
courses or visit student teachers (Mentors). The purpose of the conference was to provide feedback about the
performance of students in their teaching majors on the TWS. Data was provided for them so they could compare their major students’
performance with that of all ESU students, elementary students, and all
secondary students. The data and score sheets as feedback to
individual University supervisors is being used and supervisors will be asked
for evaluation of the process in the Spring 2003. The 16 faculty members representing all
of the departments in which methods courses are taught across the campus are
supportive of the Teacher Work Sample both as an instructional tool for their
students as well as its application as an instructional improvement strategy for
the University’s teacher education programs. The subject matter methods
teachers are actively promoting TWS in the specific methods courses taught in
each discipline. We in the Teachers College continue to be impressed with
their acceptance of and enthusiasm for the Teacher Work Sample, and believe that
with each passing semester their ability to integrate the six work sample
factors into their courses will improve. Furthermore, the Dean of LA &S
continues to be a solid promoter of TWS as a tool for improving instruction and
improving and tracking student performance. Mentors and teacher education faculty
have been asked to turn in references, resources, teaching aids, materials, and
instructional techniques related to assessment and evaluation. Knowing what they teach about assessment/evaluation and how they teach it
can be a valuable shared tool within the teacher education at ESU. Scoring
the TWS Rater training sessions are conducted each semester at ESU. Raters are teachers and faculty recruited from diverse professional backgrounds. The training process involves four stages: (1) an orientation to the TWS; (2) rater training (including practice scoring); (3) estimating reliability (retraining if necessary), and (4) individual rater scoring. Exemplars are critical to the training process. Analytic and holistic exemplars are being developing for future rater training sessions. In fall semester we experimented with pair scoring and determined that after scoring a student’s work individually, the opportunity to critique and discuss was valuable. The raters were under no obligation to change scores as a result of any discussion. As a point of interest, there were rater pairs which, independently, both
rated work samples in the high nineties (on ESU’s 100 point scale) and other
pairs who also rated different student work in the forties. At the present time ESU has trained and used over fifty well-trained and
experienced raters who have been recruited from the public schools, liberal arts
faculty, and elementary and secondary teacher education faculty. They comprise a valuable resource pool.
Future rater groups will be drawn from experienced raters in this pool
together with an infusion of new untrained raters from the public schools and
the university. Research
and TWS/PDS
Emporia State University continued to conduct research during Spring and Fall
2002 to compare differences in TWS performance between PDS and “On-Campus”
candidates. Comparisons are made on
learning impact indices as well as scores on individual rubric factors. The
study uses a matched pair design. There
is also research underway to correlate performance on TWS with other available
indices. No correlation was found between scores on state required writing ability
tests and TWS scores. This
challenges the allegation that students who write better, score higher on TWS. Research at ESU shows a lack of correlation between Scores on Work Sample
factors, leading us to believe that each factor may measure a unique quality,
knowledge, or skill. The Crocker Validity Study shows that faculty who have served as raters at
ESU believe that TWS is a valid, meaningful assessment which is related to
INTASC standards. A Student Perception Survey indicates that (1) students like the yardstick of
using student gain scores and objective mastery indices to gauge the impact they
are having on their pupils. They
also indicated that (2) they like aspects of being held accountable for
individual learners, which the structure of the TWS provides. Research on the Fall 2002 TWS performance of PDS students and students in the
regular on-campus ESU preparation program showed a slight (but statistically
significant) advantage for PDS students in overall scores and student gain
scores. Research on differences between elementary and secondary student TWS shows
that elementary candidates outscore secondary candidates on the total TWS score,
but this variance is mainly due to elevated scores by elementary candidates on
Factor Six, the reflective piece. Curriculum
Development ESU has finished the 2nd
year of data collection on the TWS. The
data collected has not only been instrumental in driving the development of the
TWS prompt and rubric, but it has led to classroom change. ESU’s
award winning program is comprised of dedicated faculty who value the formative
data feedback given by the TWS data. After
scoring the Spring 2002 TWS, elementary faculty who had served as scorers made
recommendations to their colleagues for changes in classroom activities,
procedures and curricular integration, without the need for using standard
curricular change mechanisms. Over the past two years, faculty have developed
and implemented course specific methods to introduce the TWS to their students,
and tie it into what was being learned in the classroom. In addition, Teacher
Work Sample evaluation factors have been aligned with standards from the
National Council for the Accreditation of Teacher Education, and TWS data
collection and analysis continues to be an on-going part of the curriculum
development process for accreditation. Dissemination ESU faculty presented the ESU TWS
model to the KSDE in spring 2002. In
fall 2002 the Kansas State Department of Education developed a model for TWS
based heavily on the ESU model. They
will use the KSDE TWS to assess teacher performance during the three-year
conditional licensure in order to determine the candidate’s eligibility for
the five-year professional license. The former ESU Associate Dean, Jerry
Long, has been hired by the KSDE to assist them in their development and
application of the KSDE TWS model. This
assures continuous connection with the ESU philosophy and expertise. ESU
faculty presented TWS information at KACTE and Kansas Coalition of Professional
Development Schools in Fall 2002 and Spring 2003. The Associate Dean, Phil Bennett,
made a presentation in Oklahoma for the State Department of Education to
familiarize university faculty there with the implementation of a TWS based
performance assessment design. The Associate Dean also made a
presentation in Kansas City sponsored by the Jones Institute at ESU to
familiarize university faculty from Kansas and Missouri with the implementation
of a TWS based performance assessment design. Five ESU faculty, including the LA
&S Dean, are presenting at AACTE on the topic of the use of TWS as part of
meeting requirement of NCATE assessment. Looking
Ahead We are continuously refining the processes for preparing our candidates,
training TWS raters, and integrating performance results into decision making.
The following activities have occurred or will occur during the 2001-2002
academic year. TWS
Training Teachers: In spring of 2002 ESU completed
training of one hundred thirty area public school teachers for their role as
cooperating teachers of candidates who will be writing work samples in the
coming year. The mentoring,
support, and assistance that these teachers can now give will be invaluable to
our student teachers. In the fall of 2002 we trained
fifty-eight additional cooperating teachers from Emporia, Topeka, and the Kansas
City area for their role. This
brings the grand total of public school teachers, administrators, and university
personnel (including supervising teachers) who have received training related to
the TWS since the grant began at ESU to well over 600. In late April 2003, a rater training
session and TWS scoring session will be held with approximately 30 faculty and
teachers scoring more than 140 work samples. Students:
In addition to TWS training as a part of teacher preparation courses,
during the first half of the Spring semester 2002, each elementary and secondary
student teacher was given the opportunity to use an on-line TWS help line during
student teaching as they write their work samples. Curriculum
Development All
teacher candidates:
Since the elements of TWS are now thoroughly integrated into the
elementary and secondary teacher education programs, a Matrix of TWS Knowledge
and Skills is under development in which elementary and secondary teacher
preparation faculty will identify the specific courses in which the TWS
knowledge and skills are covered. Furthermore,
the instructors will also analyze the “coverage” as being introductory,
developmental, or application. Thus,
the alignment of TWS knowledge and skills and the ESU teacher preparation
program can be tracked and analyzed. Use
of the matrix has great implications for reporting on student performance to
NCATE and for internal curricular assessment and alignment. Secondary:
Pre-student teaching students in a generic methods course are introduced
to the elements of the TWS and each developed a teacher work sample based upon
their observation experience. The
TWS was integrated into the fabric of the course and served both as a focal
point for delivery of instruction and as a culminating activity for much of the
instruction in that course.
Instructors in specific content methods courses continue to improve the
integration of various TWS skills and knowledge into their courses. Elementary:
A continuous process beginning two years ago for integrating and
strengthening TWS factors and elements into four semesters of on-campus
pre-student teaching classes for non-PDS students has resulted in major changes
in classroom activities. These students are now familiar with the format and intention
of the TWS and are quite capable of building a TWS during Student Teaching.
Training in TWS for elementary students in the PDS program has
accelerated and the work samples done in the fall of 2002 demonstrated even more
improvement in all six factors of the TWS over work samples completed in the
fall of 2001. Research
Fall, 2002 The previous academic year has been a busy one for the personnel involved in the Renaissance Teacher Work Sample (TWS) project at ESU. In addition to attending Renaissance meetings and grant-related training in St. Louis, training and implementation of the TWS concept has been an on-going activity gaining momentum and stature at ESU and within the State of Kansas.
Spring, 2002 The Teacher Work Sample (TWS) at Emporia State The TWS is a requirement for ALL candidates in the teacher education program. All elementary and secondary student teachers develop a TWS as a part of the requirements for student teaching. Mentoring Program Report During the fall semester, Dr. William Samuelson held conferences with all of the Liberal Arts and Sciences and Business faculty who teach departmental methods courses or visit student teachers. These conferences were a follow-up to the group training that was done in April 2001. The primary rationale for the fall conferences was to introduce the Mentor faculty to the last revision of the prompt and rubric and to clarify administrative issues. Scoring the TWS Rater training sessions are conduced each semester at ESU. Raters are teachers and faculty from diverse professional backgrounds. The training process involves four stages: (1) an orientation to the TWS; (2) rater training (including practice scoring); (3) estimating reliability (retraining if necessary), and (4) individual rater scoring. Exemplars are critical to the training process. Analytic and holistic exemplars are being developing for future rater training sessions. Research and Accountability TWS/PDS
Research Looking Ahead We are continuously refining the processes for preparing our candidates, training TWS raters, and integrating performance results into decision making. The following activities have occurred or will occur during the 2001-2002 academic year. Teachers:Curriculum Development All teacher candidates:Dissemination
Fall, 2001 Teacher
Work Sample
The TWS will continue to be a
requirement for ALL elementary teacher education candidates in PDS schools. In
addition, ALL elementary teacher education candidates in non-PDS schools will
pilot the TWS before it becomes a requirement next Spring. In July and August, 2001,
thirty-eight Professional Development School mentor teachers were trained in
using the Teacher Work Sample with P. D. S. interns. The mentors learned strategies for assisting the interns with
instructional design and assessment strategies. Training was also provided in rating Teacher Work Samples using products
developed by Emporia State interns during the spring, 2001 semester according to
the Emporia State rubric that was revised in June. During the fall, 2001 and spring,
2002 semester, all elementary and secondary students in methods classes and in
Professional Development School assignments will develop a practice Teacher Work
Sample to become familiar with the process and with the rubrics which are used
in the Emporia State process.
All elementary and secondary
student teachers will develop a Teacher Work Sample as part of the requirements
for student teaching during the spring, 2002 semester. Margaret Davidson, a fourth grade
teacher in the Butcher School, the laboratory school for Emporia State, is
creating a Teacher Work Sample to model for elementary education students.
The Teacher Work Sample created by Ms. Davidson aligns with one of the
Kansas standards for social studies. Ms.
Davidson pre-tested third graders to create baseline data and designed the
Teacher Work Sample over the summer. She
is teaching the fourth graders in her class this fall and will discuss the
process of making instructional decisions relating to the Teacher Work Sample in
elementary methods classes this fall. Information about the use of the
Teacher Work Sample as a performance assessment tool in the elementary
Professional Development School program at Emporia State will be shared in a
presentation by Dr. Tony Ambrosio and Dr. Larry Lyman at the Kansas Professional
Development Schools Alliance conference in Olathe, KS on September 24. Accountability
and Research
Faculty working with students who
will be required to create a Teacher Work Sample have been trained in the
process and philosophy of the Teacher Work Sample. Suggestions from faculty and from mentor teachers who worked with the
process during the spring, 2001 semester were used to revise the rubric that
will be used for fall.
Additional training of cooperating
teachers will be provided during the fall semester for teachers who will be
working with methods students and student teachers who are creating Teacher Work
Samples.
Student teaching supervisors and
university faculty gathered to develop a series of research proposals involving
the TWS. Participants identified links between PDS program learning/teaching
outcomes and TWS indicators. The research is scheduled to be completed during
the 2001-2002 academic year. The
Use of TWS as Measures for NCATE Standards Faculty
and project personnel met in August to identify links between TWS indicators and
specific NCATE performance criteria. Faculty
will be surveyed in the fall about specific classroom activities that address
each link. LOOKING AHEAD Spring 2002
Contact person: Phil Bennett <bennettp@emporia.edu> Spring 2001
Fall 2000
|
||||||||||||||||||||||||