ITQ RTWS

California State News

Renaissance Partnership Members

California State University
Central Unified School District
California Business Furnishings

Teacher Work Sample (TWS)
Based on the success of implementation of two cohorts in using the Teacher Work Sample (TWS) to drive its pre-service curriculum and to assess its students on recognized teacher-skills, the Kremen School of Education and Human Development at California State University, Fresno, is piloting a revised TWS in the much larger non-cohort teaching credential program. The revisions made to the Renaissance-generated TWS have been relatively small — the addition of a content analysis section and a classroom management plan, as well as an expansion of existing prompts to allow for more graphic presentations under Contextual Factors and Analysis of Student Learning. All original standards and rubrics were retained; the prompts were widened. While the original piloting ECE cohort will continue with the original prompt this year, the revised Teaching Unit will be piloted by both seventy students in the Multiple Subjects and Single Subjects programs. Local validity and reliability studies will be done using the Renaissance Credibility Manual. It is hoped that by fall, 2004, the performance of all 2,000+ credential students at CSU, Fresno, will be, in part, assessed by the new Teaching Unit.
A separate revision of the TWS was done last year by Special Education faculty Hong Shen and Craig Minor. Their modified TWS meets the state requirements for both Level I and Level II coursework and is applicable to students training to work with children with both mild/moderate and moderate/severe disabilities. Please contact those professors if you desire additional information on their modifications, infusion into coursework, and the use of the revised sample as a summative evaluation tool.

Mentoring
Our experimental mentoring delivery system — described in our spring, 2004 semester as a “Drop-In Center” — did not increase students’ use of content- or process-related coaching. Because cohorts have required students to complete a TWS as part of their initial coursework, most final student teachers reported that they did not need assistance the second time the TWS was completed. That may be true of process-related issues, but content issues continue to be a challenge to our students. We continue to pursue a system that is user-friendly, but that provides content input, even to those who are not aware they need it! The new Renaissance Mentoring Manual will, no doubt, be useful here.

Accountability Systems
CSUF’s Renaissance Project Assessment Coordinator Robin Chiero, Ph.D. also serves as the coordinator of our NCATE accreditation scheduled for 2006. Her expansion of our data collection and storage capacity will provide schoolwide access to student data, including both NCATE- and state-mandated performance data. We continue to gather credibility data on our Teacher Work Sample pilot begun four years ago and on the two revised editions currently being piloted in the general preservice training programs.

Spring, 2003

California Update: State Adopts Its Own Summative Assessment Model

Based on the success of the Teacher Work Sample (TWS) in enhancing the preservice training of teacher candidates enrolled in the Early Childhood Education emphasis credential program at California State University, Fresno, the use of the Teacher Work Sample as one of the means of assessing preservice teacher performance has expanded this year to include a second cohort programs, raising the number of candidates completing the TWS to 90 annually. In addition, our Special Education Credential Program will be piloting an adaptation of the TWS in their fieldwork with children with mild-to-moderate and moderate-to-severe handicaps. Although it was hoped that by year’s end, the general Multiple Subjects Credential Program and the Single Subject Credential Program — collectively graduating 1500+ teachers a year — would use the TWS as a teacher performance assessment, current state legislative mandates may prevent us from using the TWS in its pure form as a summative assessment tool. The state of California has designed its own summative assessment — the California Teacher Performance Assessment (TPA) — that is aligned with the state’s Teacher Performance Standards. Even though we believe the TWS to be aligned with the state’s standards, to be a more valid and rigorous assessment of preservice teacher knowledge and skills, and to be logistically more practical than the four-task state version, we are mandated to use the state’s instrument as part of its high-stakes assessment system. The piloting ECE Program is committed to continuing to use the TWS as a summative assessment, but at a schoolwide level, our challenge is now to find a way to adapt the TWS to include the four tasks required by the state system. If impossible, the TWS may well be used as a formative assessment tool to prepare students for the summative state examination. Certainly the lessons learned in developing, implementing, and assessing the TWS as a formative and summative assessment and teaching tool will be applied to the implementation and assessment of the TPA.

Teacher Work Sample (TWS)

While the Kremen School of Education and Human Development develops an implementation system for the state’s mandated assessment, the ECE and Reentry cohorts will continue to use, refine, and be guided by the Teacher Work Sample. By expanding the use of the TWS to another cohort program, faculty will be able to determine if the TWS can be used with different delivery systems. Although all students will document their performance using the TWS in final student teaching, initial exposure to the TWS will differ between cohorts. For example, the ECE program requires that students complete their first Teacher Work Samples in their curriculum class which includes a ten-week field experience; students apply the elements of the TWS to real children in an actual classroom. That delivery system will differ from the other cohort which does not require concurrent enrollment in a student teaching placement. Common data, scenarios, and modeling are being developed to familiarize students with the skills and processes necessary to successfully complete a TWS in their final student teaching placement. This local research will, no doubt, have implications for overall program delivery and performance assessment.

The Work Sample Checklist developed by faculty member in charge of final student teaching assignments in the Early Childhood Education Program, Marilyn Shelton, Ph.D. continues to bridge the gap between preservice training and actual professional service. By creating a checklist that demands accountability to the student teacher’s master teacher and university supervisor, student teachers in their second final student teaching placement must attend to each of the standards required in the Teacher Work Sample, but without the 20 page+ written document. The design of Dr. Shelton’s transitional tool recognizes the elements of the TWS and the process of planning and reflection as essential to good curriculum design and teaching while acknowledging that real teachers do not and cannot write a 20-page paper for each instructional unit they plan! It is hoped that “habits of mind” will be developed with these three experiences with the TWS methodology. Post baccalaureate surveys and interviews now being planned will gather data as to the influence of the TWS in planning and assessing student learning after one and two years of professional service.

Mentoring

In anticipation of expanding the program to all six “pathways” to a California Multiple Subject, Single Subject, and Special Education Credentials and 2000+ students , the CSUF Renaissance Project piloted a new delivery system for mentoring students completing a Teacher Work Sample in final student teaching — a drop-in style Mentoring Center. Liberal Studies, foundations course, and methods class faculty as well as fieldwork supervisors and site-based Master Teachers were available at the Center to assist students in both content-related and TWS process-related issues. Unfortunately students did not avail themselves of the opportunities for assistance, just as they had not taken advantage of mentors through other delivery systems. Data have been

gathered from students who completed their TWS in fall, 2002, regarding mentoring opportunities. After analysis we will hopefully be able to determine why there appeared to be a lack of interest in this service. Dr. Jean Behren, our Coordinator of Mentoring and Teacher Work Sample, will no doubt use some of the ideas in the Project’s new Mentoring Manual she helped to develop.

Accountability Systems

CSUF’s Renaissance Project Assessment Coordinator Robin Chiero, Ph.D. also serves as the coordinator of our NCATE accreditation scheduled for 2005. Her expansion of our data collection and storage capacity will provide schoolwide access to student data, including performance data.

Fall, 2002

Teacher Work Sample (TWS)
The Teacher Work Sample continues to influence university classroom instruction as faculty members score and, more importantly, analyze student performance on the various required elements of the TWS. By comparing Work Samples created two or even one year ago with those submitted this past spring, we know that preservice instruction in student assessment and in the identification of appropriate content, pedagogy, assessment tools, and resource materials based on the teaching context and the learner has significantly improved. Historically our students have been able to identify student needs and skills, as well as critical contextual elements within the teaching environment, but they have not fared as well in addressing the question of “And so what?” — that is, identifying the implications of the teaching context to instruction. We are seeing improvement in that area and, now that coursework in special education mainstreaming and second language acquisition are being considered as prerequisites to the credential program instead of post-credential requirements, we anticipate documentation of higher levels of learning for English Learners and special needs students via the TWS during both initial and final student teacher fieldwork experiences.

By expanding the use of the TWS to other cohort programs, faculty will be able to determine if the TWS can be used with different delivery systems. Although all students will document their performance using the TWS in final student teaching, initial exposure to the TWS will differ between cohorts. For example, the ECE program requires that students complete their first Teacher Work Samples in their curriculum class which includes a ten-week field experience; students apply the elements of the TWS to real children in an actual classroom. That delivery system will differ from other cohorts who do not require concurrent enrollment in a student teaching placement. Common data, scenarios, and modeling are being developed to familiarize students with the skills and processes necessary to successfully complete a TWS in their final student teaching placement. This local research will, no doubt, have implications for overall program delivery and performance assessment.

In charge of final student teaching assignments in the Early Childhood Education Program, Marilyn Shelton, Ph.D., has designed a “Work Sample Checklist” meant to bridge the gap between preservice training and actual professional service. By creating a checklist that demands accountability to the student teacher’s master teacher and university supervisor, student teachers in their second final student teaching placement must attend to each of the standards required in the Teacher Work Sample, but without the 20 page+ written document. The design of Dr. Shelton’s transitional tool recognizes the elements of the TWS and the process of planning and reflection as essential to good curriculum design and teaching while acknowledging that real teachers do not and cannot write a 20-page paper for each instructional unit they plan! It is hoped that by infusing the knowledge and skills necessary to successfully design, facilitate, assess, and document student learning into all preservice coursework, by providing direct instruction in the process of the TWS in coursework with concurrent fieldwork, by providing content and process mentoring during the more independent TWS completed during final student teaching, and, finally, transitioning to the more authentic “Work Sample Checklist”, graduates will maintain and expand upon the attitudes, skills, and processes taught and documented through the Teacher Work Sample in real life professional situations. Future post-graduate data will shed light the impact of the TWS and CSUF’s delivery systems on induction-level professional service performance.

Mentoring
In anticipation of expanding the program to all California Multiple Subject, Single Subject, and Special Education Credentials programs (2000+ students), the CSUF Renaissance Project is piloting a new delivery system for mentoring students completing a Teacher Work Sample in final student teaching. A drop-in style Mentoring Center is being organized during the late afternoon and evening hours one day a week. Liberal Studies, foundations courses, and methods classes faculty as well as fieldwork supervisors and site-based Master Teachers are staffing the Center to assist students in both content-related and TWS process-related issues. We are hopeful that this will increase both student participation and faculty participation in the mentoring process, and provide a practical model for future expansion.

Accountability Systems
Assistant Professor Robin Chiero, Ph.D. has recently been named the CSUF Renaissance Assessment Coordinator. Her background in teacher education, the Teacher Work Sample, program evaluation, and advanced technology systems will allow CSUF to expand the scope of their accountability system and strengthen its potential for ongoing program evaluation. Because California is currently field-testing a statewide Teacher Performance Assessment (TPA), education faculty at CSU, Fresno will soon be comparing the performance of the state’s instrument to the Teacher Work Sample, an instrument that appears to be a valid performance assessment of the California’s Teacher Performance Expectations, the “bottom line” criteria for local choice of a TPA. A decision may be made to use both instruments in the spirit of “multiple assessments,” to replace the TWS with the California TPA, or, as is allowed by the state, to replace California’s TPA with the Renaissance Project’s TWS at the local University level.

Meanwhile, the impact of the Teacher Work Sample on teacher performance continues to be measured through interviews with Master Teachers who supervise final student teaching, and by analyzing student performance relative to the NCATE standards, the Standards for the California Teaching Profession, and, in the case of Early Childhood Education Program participants, the standards of the National Association for the Education of Young Children (NAEYC). Alignment with these standards has resulted in plans for an expanded professional portfolio for use as a means of documenting preservice performance, for use in job interviews, as an indicator of entry level performance into local induction programs (BTSA), and as a means of program evaluation. Performance information will include exit interviews with student teachers, providing self-assessment information relative to the professional standards. The ECE program will continue to annually survey its graduates and graduates’ employers to determine performance levels, although information on and from graduates will be expanded to two years following graduation from the program. These levels will be compared to data collected in 1999-2000. An analysis of the results of graduate and employer surveys will continue to inform our practice at the University.

Spring, 2002

In its fourth semester of implementation at CSU, Fresno, the Renaissance Project is being expanded from its original pilot Early Childhood Education cohort group of sixty graduates annually and five university and clinical program faculty to include two additional California Multiple Subjects Credential cohort programs, an expansion that will result in 110 graduates annually having completed a Teacher Work Sample (TWS) as part of their preservice training and the inclusion of an additional fifteen faculty. These two additional cohort programs have their own unique emphases - middle school training and a district-based program for re-entry students - as well as unique program delivery systems. Using the successes and failures of the ECE pilot group, these cohorts will adapt the TWS to meet their unique program needs while providing a common school-wide element in preservice training and documentation. Faculty is interested in if the TWS can be adapted to meet the needs of diverse program delivery systems and grade/age level foci while providing a common performance assessment of Multiple Subject Credential graduates and maintaining its Project integrity. It is hoped that the strategies used this spring to expand TWS implementation two-fold, will provide a model for expansion next year to the much larger Option I credential group which graduates 1,500 students annually and involves over 150 university and clinical faculty.

Plans for Spring, 2002

The Early Childhood Education Program and the two cohort groups joining the Renaissance Project at CSU, Fresno, - Option IV and Block A -and representative program administrators from the larger Option I will together:

refine a common Master Teacher orientation;
work with an enthusiastic Advisory Group - a group that includes Renaissance Partner Central Unified School District as well as three other local districts - in assuring a seamless transition from preservice training to California's beginning teacher induction program;
work in job-alike groups to train university and clinical faculty cohort teams new to the Renaissance Project to use the TWS as a curriculum guide and a source of documentation;
establish interrater reliability within the expanded group of faculty-raters; and
explore alternative mentoring systems that can accommodate a much larger number of participating students.

In addition, the Early Childhood Education Program will gather survey data from TWS-trained graduates and their employers to determine program satisfaction and postgraduate professional performance. The results of an analysis of the data will be used to inform program practice.

Teacher Work Sample (TWS)

By May, 2002, 65 ECE students will have completed both an initial Teacher Work Sample as part of the requirements for their Integrated Curriculum class and a second Sample in their final student teaching fieldwork. Preliminary data indicate an improvement between the first and second Samples, particularly in the design of valid and reliable pre- and post-assessment instruments and instructional decision making. A comparison analysis is being conducted of Samples produced with mentoring and without to determine the impact of the CSUF mentoring system.

Mentoring

Of the 25-30 final student teachers completing a Teacher Work Sample each semester, five students are being mentored on an "on-call" basis. Analysis of the contact logs being maintained by both student teachers and mentors indicate that while TWS process mentors are being used at least once during the semester, arts and science faculty are rarely being called upon for content expertise. The data gathered through student focus groups and phone interviews will assist us in determining how we can improve this Renaissance Program component. This is a top priority as we quickly approach full and expanded implementation of the Renaissance Project.

Accountability Systems

The Early Childhood Education Program will gather survey data from TWS-trained graduates and their employers to determine program satisfaction and postgraduate professional performance. The survey questions reflect the skills required in the TWS; the results of an analysis of the data will be used to inform program practice.

Fall, 2001

California State University, Fresno, is in their second year of Renaissance Project implementation. Currently being piloted by the Early Childhood Education Program, the Teacher Work Sample (TWS) was completed in each of two fieldwork placements by each student enrolled in final student teaching during the spring and summer, 2001, semesters. The TWS served as one of several sources of documentation of students’ competency. Those who completed their final student teaching in the summer had the advantage of having completed an initial TWS in the previous semester as part of their curriculum class and concurrent initial fieldwork placement. Students were encouraged to synthesize information and skills gained in their methods and foundation coursework with that gained in their curriculum class experience, to apply that information in their fieldwork, and to document that application using the TWS. The ECE program believes that by “walking students through” the TWS in the more controlled university classroom setting, students will be more able to use the TWS as a planning tool in their final pre-service experience. Evaluation of the TWS is done using the analytic rubric in the initial experience and the holistic rubric in evaluating the two final products. It is hoped that the specificity of the analytic rubric in initial scoring will provide the student with more detailed assessment information, as well as to faculty working to revise program curricula. For example, although the holistic rubric indicated that students performed generally well in the “context” section, the analytic rubric revealed an important and pervasive lack of knowledge as to how to apply context to lesson design. That information, when shared with students and faculty, will hopefully motivate students to more competently address that point and will inform faculty as to the need to better facilitate student learning in that area.

Plans For Fall, 2001

California State University, Fresno, is preparing this semester to expand the number of pre-service participants involved in the Teacher Work Sample. In anticipation of adding two cohorts of twenty-five students in spring, 2002, to the ongoing TWS-driven program of the 30- member Early Childhood Education cohort, work this fall is focusing on:

the inclusion of the TWS in an expanded portfolio documentation of standards required by NCATE, the California Commission on Teacher Credentialing, and the NAEYC;
a major revision of orientation materials for Master Teachers hosting participating students in their initial and final student teaching fieldwork;
an expansion of the participation of university-based mentors from the School of Arts and Sciences as well as inclusion of methods course faculty from the School of Education;
training university colleagues in the TWS protocol;
and rigorous data collection from graduates who completed the TWS and are now teaching in California classrooms.

On-Going Discussion

On-going discussions include

the development of a more effective mentoring delivery system,
revision of curricula to empower students with the skills, knowledge, and attitudes necessary for successful completion of the TWS—especially in the area of student assessment and the application of contextual factors to pedagogy; and
developing plans for expanding the program from three small cohort groups to the much larger program of 1,000 students in fall, 2002.

Teacher Work Sample (TWS)

Documentation of student teaching skills via a traditional “Resource Unit” were evaluated by CSUF program teaching and clinical faculty using the Teacher Work Sample rubrics to establish a baseline performance level to which subsequent performances using the Teacher Work Sample could be compared. This spring ten student teachers completed the first of two Teacher Work Samples, one sample being required for each of two, seven-week placements. Five of those student teachers received mentoring, five did not. Each TWS generated in students’ first placements were evaluated by three faculty members using the TWS rubric in order to establish interrater reliability and to score performance of the university students. The identification of exemplary products or product sections will be used as exemplars in conjunction with the TWS rubrics. Teacher Work Samples with mentoring will be compared to Samples without the mentoring variable to determine over time the impact of mentoring on TWS quality. In addition to the products generated by students in their final practicum, thirty-three second-semester ECE students are using the TWS to guide the documentation of their initial fieldwork experience. Practice with the TWS process, it is hoped, will enhance performance in final student teaching.

As important as documenting student learning, the TWS has made a profound impact on ECE preparatory class curricula. Initial data indicate a greater focus on the design of valid and reliable formative and summative testing, and on the use of assessments to drive curricular design. The task takes on issues of professional ethics when accreditation agencies are at odds as to appropriate assessment and instruction for young children, the focus of the piloting teacher preparation program.

Mentoring

Mentoring teams using five Master Teachers from partner institution, Central Unified School District, University teacher preparation faculty, and Arts and Science faculty are “on-call” to the five final student teachers placed in Central Unified School District. Contact logs are being kept to document both frequency and content of mentoring tasks. Each of the five Master Teachers has, herself, completed a Teacher Work Sample; such experience provides authentic assistance to student teachers in the pilot. Mentors from within and from outside the University continue discussion as to the roles of mentors, and the availability and selection of Project mentors.

Accountability Systems

The impact of the Teacher Work Sample on teacher performance is being measured through interviews with Master Teachers who supervise final student teaching, and by analyzing student performance relative to the NCATE standards, the Standards for the California Teaching Profession, and the standards associated with the ECE program—the National Association for the Education of Young Children (NAEYC). Alignment with these standards has resulted in plans for an expanded professional portfolio for use by program graduates.

Performance information will include exit interviews with student teachers, providing self-assessment information relative to the professional standards. The ECE program will continue to annually survey its graduates and graduates’ employers to determine performance levels, although information on and from graduates will be expanded to three years following graduation from the program. These levels will be compared to data collected in 1999-2000. An analysis of the results of graduate and employer surveys will inform the practice of the University program.

TOP