The report Getting to Teacher Ownership: How Schools are Creating Meaningful Change and its supplementary materials are based on a multi-method study conducted by AISR. The study employed qualitative case studies, a teacher and administrator survey, and school-level quantitative data. We integrated in-depth qualitative data and analysis with the broader perspective provided via quantitative data and analysis.
With the assistance of three nonprofit organizations working in partnership with the Los Angeles Unified School District (LAUSD), we identified and invited three high schools currently implementing some combination of community schools, Linked Learning, and Promise Neighborhoods as the foci of our case studies:
- STEM Academy of Hollywood at Bernstein High School, a Linked Learning and Promise Neighborhood school;
- Social Justice Humanitas (SJH) at Cesar Chavez Learning Academies, a community school situated within the Los Angeles Promise Neighborhoods boundary; and
- Los Angeles School of Global Studies at Miguel Contreras Learning Complex, a Linked Learning school.
Of the 36 Linked Learning pathways operating within the LAUSD in 2014-2015, 7 Los Angeles Promise Neighborhood (LAPN) schools, and 13 community schools working with the Los Angeles Education Partnership (LAEP), we identified potential case study schools. Sites were selected based on their demonstrated commitment to implement the particular approach (according to partner organizations). Further, we considered sites that served a population of students representative of the district’s high school population as a whole and had graduated at least one cohort of students. See below for the demographics of the case study schools.
Administrators of each identified case study site were sent a project description and an invitation letter via email. Follow-up phone conversations enabled administrators to ask additional questions regarding the project and participation.
Data Collection Instruments
Data collection and theoretical framing were based on a thorough review of the literature on teacher ownership and related topics. One-on-one interview and observation protocols were designed based on this review. Interview and observation protocols probed teachers’ understanding of improvement goals and student expectations, opportunities for teachers to work collaboratively, opportunities for teachers to share knowledge and practices, opportunities to share knowledge and experiences, opportunities to lead improvement efforts, and the conditions and resources that enabled and/or prevented these practices.
During fall 2015, we conducted 28 interviews and 30 classroom and professional learning observations during two- to three-day site visits. Teachers, administrators, counselors, and on-site community partners participated in one-on-one, in-person interviews. Interview protocols were designed to allow participants to openly discuss their experiences at their schools.
One research team member was assigned a “site leader” role and coordinated the site visit. A few research team members were assigned to one study site, while other team members participated in all three site visits so that they could gain a cross-site understanding of participating schools.
Across the three case study sites, we worked with the principal to identify “lead teachers”: teachers who form part of the leadership team; have been involved in the development of the initiative or improvement effort, its leadership, and/or its evaluation; and/or hold formal or informal responsibilities at the study site. We aimed to interview three to four “lead teachers” and three to four additional teachers at each site and to observe formal and informal meetings teachers held with their peers. Interviews and observations averaged forty-five minutes in length. All interviewees were assured anonymity.
Through interviews and observations, we aimed to gain a sense of what teacher ownership looked like within the school setting. In particular, we aimed to understand whether and how ownership extended beyond teachers’ feelings about their own craft and classroom to the overall school environment and to a larger network of schools.
The research team created detailed field notes and voice-recorded all interviews in order to have an accurate record of participants’ perspectives. We also collected school-level data regarding school staffing, student characteristics, school conditions, and outcomes to provide important contextual information in which to situate each school’s implementation and improvement efforts.
A survey was designed and administered that aimed to measure school climate and the extent to which teachers felt they understood the approaches being implemented at their school site, had opportunity to learn about the approaches, and had opportunity to direct and lead implementation efforts at their school site. A thorough scan of the literature and existing survey instruments assisted us in identifying instruments that explored topics such as collaboration, trust, professional development, school climate, and professional learning opportunities. These validated instruments included: WestEd’s California School Climate Survey and California School Staff Survey; My Voice, My School Teacher and Principal Survey from The University of Chicago Consortium on School Research; Los Angeles Teachers Tie Project Survey; The New Teacher Center’s Teaching, Empowering, Leading and Learning Survey (TELL); School Culture-Climate Surveys developed by Stanford’s John Gardner Center; and the Los Angeles Unified School District’s School Experience Survey. We modified questions from the instruments identified to meet our research needs and designed questions to fill research gaps. (See below for a sample of survey questions.)
A parallel survey was designed and administered that sought the perspective of administrators on these same teacher-related topic areas.
The survey was piloted at a non-participating LAUSD high school and modifications were made based on teacher feedback.
In addition to ownership-related constructs, the survey included a series of demographic questions about years of teaching experience, teachers’ roles and responsibilities (outside of classroom responsibilities), course(s) taught, ethnicity, age, and gender. The survey was administered online. The survey was anonymous, but teachers were required to identify their school from a list of options. Incentive for participation was comprised of a small monetary contribution to the school site for each survey submitted. Individual incentives for participation were not provided.
A total of 36 LAUSD high schools were invited to participate in the survey. In collaboration with the LAUSD and local partners, we identified a list of sites that were engaged in the implementation of the approach for a minimum of four years. A total of 346 administrators and teachers from 29 schools responded to our invitation. Eight schools did not meet a minimum response rate threshold of 10 percent, and hence a total of 21 schools were included in our final sample and analysis. Overall, a response rate of 41 percent was achieved.
A total of 190 administrators and teachers were included in our analysis (substitute teachers, counselors, and program coordinators were excluded, as well as individuals who did not complete the entire survey): 65 teachers indicated that they taught at schools implementing a community schools approach, 74 taught at a Linked Learning pathway, 67 taught at schools implementing a Promise Neighborhoods approach, and 30 indicated that they taught at schools implementing more than one of these approaches. These 30 teachers are represented within counts for each of the distinct approaches (e.g., one teacher may be included as Linked Learning and a community school teacher). Table 1 provides a description of the final teacher sample.
Student-, School-, and District-Level Data
School-level data for participating sites and overall district-level data was provided by the LAUSD. Data provided included information regarding teacher demographics, experience, attendance rates, and turnover rates for the 2013-2014 and 2015-2016 school years. Student data included suspension rates, graduation rates, A-G college preparation course completion rates, and enrollment and pass rates for advanced placement course.
Interviews were transcribed verbatim in preparation for data analysis. Transcripts, field notes, and memos were coded using Dedoose, software designed for this purpose. A coding schema was created based on the conceptual ideas introduced in the interview protocol. Additionally, codes were generated using an open coding approach that allowed new concepts to be identified and introduced. The research team met on a weekly basis to discuss emerging codes, concepts, and categories. Research team members also continued the practice of memo writing during the coding process to capture their ideas and insights regarding substantive codes and their relationship to other codes.
Preliminary reports for each site were prepared. The full team discussed preliminary reports to: 1) ensure both factual and interpretive accuracy; 2) identify missing information or ideas; 3) ensure a cross-section of perspectives among team members was represented; and 4) identify any shared themes across sites. The next phase included cross-case analyses to illuminate strategies, patterns, and principles that cut across the sites as well as discern the differences among the sites.
A write-up of findings was sent to external reviewers, and revisions were made based on feedback.
As is common with most case study research, the issue of generalizability was a concern. In particular, the extent to which case study sites were representative of the respective approaches was a potential limitation identified. There is diversity across and among schools implementing the three approaches under study, and we were well aware that three cases could not adequately capture the range of schools implementing the approaches or capture how each context shapes the experiences of teachers. By relying on quantitative data collected from a larger sample of teachers who represented the wide range of Linked Learning, community schools, and Promise Neighborhoods schools within the LAUSD, we aimed to address this limitation.
Another limitation involved the rich and expansive data collected across the three case study sites.
As a team we needed to make difficult decisions regarding what data should be included (and what should be excluded) in shaping the direction of our research findings and conclusions. Regularly held team meetings as well as internal reviews of write-ups assisted in addressing this concern.
Survey responses (primarily closed-ended responses) were entered into a data file, and analysis was conducted using SPSS statistical analysis software. Quantitative analysis included descriptive statistics, cross-tabulation analysis, tests of significance, and exploratory factor analysis. These analyses provided useful tools to understand the data as a whole; compare the results across variables and examine relationships within the data; understand the strength of particular relationships; and examine internal reliability of the theoretical constructs represented by sets of survey items.
Correlation analysis focused on seven scales based on 5-point Likert survey questions (“strongly agree” to “strongly disagree”). As shown below, each scale demonstrated excellent internal reliability as indicated by Cronbach’s alpha coefficient.
A two-way flow of information and inquiry between quantitative and qualitative analysis allowed for a synthesis of findings. Qualitative analysis and preliminary findings generated a series of inquiries that instigated additional quantitative analyses and an exploration of the extent to which findings could be generalized to the large sample of schools. Similarly, quantitative analysis and findings prompted additional qualitative inquiries and analyses that allowed us to explore a particular phenomenon in greater depth.
Securing a high response rate from teachers at participating sites presented a limitation. The research team was unable to reach out to and invite teachers directly to participate in the online survey. Rather, we needed to rely on administrators to encourage teacher participation and to share the survey link with their staff. Securing the participation of administrators and teachers at larger LAUSD schools implementing Linked Learning, community schools, or Promise Neighborhoods was particularly challenging. As such, the sample of survey respondents is not representative of all teachers within schools implementing one or more of the approaches under study.
While the survey instrument was designed to gather information on a range of topic areas we anticipated were related to teacher ownership (e.g., school climate, leadership, learning opportunities) and write-in opportunities were provided, the data that were produced lacked the depth necessary to fully understand the complexity of the construct. Qualitative data assisted in addressing this limitation.