Target audience

This resource is designed for school principals, executive teams and school staff. Directors, Educational Leadership (DELs) and Principals, School Leadership (PSLs) may utilise this resource to develop data literacy skills with school staff to improve engagement with the school excellence cycle and effectively evaluate curriculum implementation.

What and why

Curriculum reform provides an opportunity for schools to place curriculum at the centre of school planning. Effective curriculum implementation drives student growth and attainment, and school improvement. This requires:

• the development of professional capability to effectively use data to monitor and evaluate curriculum implementation.
• evaluating curriculum implementation activities that focus on the Leading, Teaching and Learning domains within the School Excellence Framework.
• a focus on curriculum implementation in areas such as:
  • educational leadership,
  • staff capabilities,
  • learning and development,
  • improvement goals,
  • policy,
  • data skills and use,
  • classroom practice, and
  • assessment.

This resource was developed as a guide for schools to understand and build data analysis capabilities to effectively evaluate curriculum implementation, highlighting the importance of:

• understanding data types (qualitative and quantitative) including strengths and limitations
• best practice data analysis (bias, collection and collation), and
• triangulation.

When and how to use

Understanding the principles of evaluative thinking and applying effective evaluation processes enables schools to investigate their curriculum implementation initiatives in a meaningful way. Effective curriculum implementation is an iterative and continuous improvement process, occurring simultaneously for different syllabuses as they are released.

This resource can be used to:

• implement effective data collection, collation, analysis practices to support school leaders to monitor, review and adjust supports for curriculum implementation within their Strategic Improvement Plans (SIPs) and implementation, progress and monitoring (IPM).
• build staff capacity to embed the use of data into their daily practice, supporting a strong culture of evaluation, building improved capabilities of using data for continuous improvement for curriculum implementation across the school.

Contact

Email questions, comments, and feedback about this resource to contactcurriculumreform@det.nsw.edu.au.

Data is defined as: measurements or observations that are collected as a source of information. (CESE, 2023). There are many different types of data that can be collected by schools to ascertain information on curriculum implementation, which may include:

• documents: units of learning, teaching and learning programs, scope and sequences, assessment schedules, work samples or assessment tasks
• observations: student achievement, lesson observations, learning walks
• administrative records: student achievement, assessment or attendance records, or staff professional learning records
• survey or focus groups: student exit slips or feedback from teachers on curriculum implementation from professional learning exit slips.

Data becomes evidence when it helps us answer a question or test whether a claim is true. (NSW DoE, 2023). Analysing student data helps teachers to identify areas of student need.

In the context of curriculum implementation, evidence provides a sound reference point for schools to reflect upon, review and refine teaching and learning to meet the needs of staff and students. Information obtained from data can inform school leadership teams to plan next steps for curriculum implementation, identifying implementation practices that are ‘on track’ or ‘off-track’, whilst also informing when staff or students are ready to undertake the next phase of curriculum.

Interpreting and analysing data effectively requires an understanding of the features of data concepts and evaluation tools. Data literacy incorporates the ability to recognise and use visual and numerical displays to describe data, and to apply this understanding to draw conclusions from the information.

To effectively evaluate curriculum implementation, school leaders and teachers may benefit from reviewing data concepts through the following resources

• CESE glossary: a glossary of data and evaluation terms.
• Evaluation resources hub resources: professional learning resources for evaluation
• Universal resources hub (staff only): evidence guides for school planning
  • Principles of collecting internal data (staff only) (DOCX 82 KB)
  • Sources of evidence (staff only) (DOCX 82 KB)
  • Quantitative data (staff only) (PDF 321 KB)
  • Qualitative data (staff only) (PDF 108 KB)

Information that can be reduced to a set of numbers, for example, where something is counted, measured or assessed. Can be used to determine averages, differences or totals (numerical data, statistics).

Benefits

• helpful to summarise large amounts of information
• tracks trends over time and highlights patterns
• can be useful when measuring change
• can be presented in an easily comparable form (such as percentage score)

Limitations

• does not provide specific details about an individual
• does not provide reasons for trends or patterns
• can give the illusion of being precise

Curriculum implementation examples

• professional learning attendance – syllabus familiarisation (evidence of activity)
• professional learning exit slips – staff knowledge/confidence to implement the new curriculum (evidence of process quality)
• pre and post-tests – unit of learning (evidence of impact)

Analysis considerations

• variability is normal, data can be influenced by unexpected factors – undertake triangulation to validate data
• confidence intervals allow for a more reliable range of values
• look at the distribution of scores
• only note the facts
• ensure consistency of data collection, recording and storage
• visually plot data to identify trends and patterns
• identify strengths, weaknesses and gaps
• sample size is important, a large sample size results in more reliable generalisations
• classes/cohorts to track data over time
• compare scores across different assessment tasks, to determine individual student growth
• be mindful when comparing achievement to external benchmarks, as these reflect the ‘average’ student
• look at growth between assessments (such as pre/post-tests)

Information from open-ended questions, observations, work samples, pictures, audio or other sources. Can be used to determine the what, how and why of an evaluation question (non-numerical, stories).

Benefits

• provides context around quantitative data
• gives insight into thoughts, feelings, opinions and experiences
• explores emerging issues and provides rich narrative

Limitations

• can be time consuming to analyse
• broad generalisations are difficult to determine, depending on the specific context
• self-report data relies on memory and can lead to social desirability bias

Curriculum implementation examples

• teacher focus groups – usefulness of syllabus familiarisation PL (evidence of activity)
• short answer survey responses – staff knowledge/confidence to implement the new curriculum (evidence of process quality)
• document analysis – unit of learning reviewed for the alignment to the scope and sequence (evidence of activity)

Analysis considerations

• thematic analysis requires identification and examination of patterns and themes in data – consider relevance to the inquiry question, what stories are emerging?
• identify strengths, weaknesses and gaps
• ensure consistency of data collection, recording and storage
• consider the statistical significance of data (i.e.accuracy of respondents and the representation of the wider population)
• avoid making generalisations from small sample sizes
• consider quantifying responses to draw comparisons across results
• keep the scope of your analysis tight to avoid being overwhelmed with information
• triangulate results with multiple data sources
• assign codes for responses to de-identify individual responses from surveys, observations or focus groups
• consider visually representing results for ease of comparison and reporting