Process evaluation and feasibility study of In:tuition, a life skills education programme for young people aged 9-14 years

Note: This report was funded and/or written by our predecessor organisation, Alcohol Research UK (ARUK).

In November 2011 CSN, a community interest company, was appointed by Alcohol Research UK to undertake an initial, independent process evaluation and feasibility study of the implementation of In:tuition.

In:tuition is an evidence based, life skills education programme for young people aged 9-14, developed by Drinkaware. The programme consists of 10 primary lessons and 11 secondary lessons. It is based upon the Unplugged programme that has good evidence of effectiveness. Alcohol is its theme but the focus is much wider, including:

• self awareness
• attitudes and behaviour
• advertising, branding and the media
• personal choices
• emotions, communication skills and assertive behaviour
• the influence of peers
• goal setting and confidence

In:tuition is also digitally based to make the learning experience relevant and motivating. Home learning tasks are included to encourage discussion with parents/carers and promote consistent messages to young people at home and school.

The evaluation looked at the implementation of the programme in the classroom, how it was used, the extent to which it was implemented as intended, its acceptability to staff and students; its relevance and appropriateness, barriers to implementation and any requirements for additional support or training.

Schools provided initial information regarding their plans for piloting the programme. Feedback was collected from staff and pupils through online questionnaires, school visits, teacher interviews and pupil focus group discussions.

The programme was piloted in 18 primary schools, 3 middle schools and 12 secondary schools recruited from across the UK. Most used either the primary or secondary lessons, but two of the middle schools used some of both. In total 20 schools piloted primary lessons and 15 piloted secondary lessons.

All the schools that piloted the programme and 4 that decided not to, provided teacher feedback through the questionnaire (30 schools) and/or interviews (26 schools); however feedback from pupils was limited to questionnaire response from 123 pupils in 7 schools and focus group discussions in 11 schools.

Only 4 schools managed to pilot all 10 primary lessons, however the majority of primary schools had piloted most of them. Often teachers who had intended piloting all the primary lessons ran out of time to fit in the last one or two. Teachers sometimes spent more than one session on a lesson in order to do it properly and get the most out of it. The primary school curriculum has the flexibility to accommodate this, but it did compound the problem of time.

Time and curriculum constraints were more of an issue for the 15 schools that piloted the secondary lessons. Time for PSHE can be very limited and programmes crowded. Five of the 15 schools, including the 3 schools that piloted all 11 secondary lessons, did so with a small targeted group of pupils, which offered more flexibility. Of the 10 schools that piloted secondary lessons within their timetabled PSHE education programme none completed all the lessons and only 3 schools used most of them.

The great majority of teachers were very positive about the programme. They agreed that it enhanced PSHE education. They liked the coherent, broadly based (life skills), interactive nature of the programme and found that it engaged pupils. They highlighted the value and appropriateness of lessons concerned with developing self awareness, understanding health and wellbeing and improving communication and decision-making.

On the whole teachers appreciated the comprehensive nature of the lesson notes, the PowerPoint presentations and the option for doing a lesson with or without the digital tool, although they also had some suggestions for improvement.

The digital tools were seen as both a real strength of the programme and a major challenge. Most teachers used at least some of the tools, but some felt they were not always essential. Both teachers and pupils identified tools that had engaged and motivated pupils and played a key role in meeting the aims of the lesson. All schools experienced some difficulties using some of the tools, often it was a problem of access through the school system, but sometimes it was a difficulty with the site, for example not allowing multiple logins.

The greater flexibility of the primary curriculum and ease of access to ICT meant that teachers piloting the primary lessons found it easier to use the digital tools as part of the programme and to adopt a cross-curricular approach. In secondary schools access to ICT facilities during PSHE education seemed to be particularly difficult even when provision in school was claimed to be good.

Those teachers who had piloted all or most of the primary or secondary lessons were more likely to identify the benefits of the programme, for example the way it supported the transition to secondary school, improved communication and assertiveness skills and even, challenged and shifted beliefs.

Pupils generally enjoyed the lessons and found them interesting, engaging and different from their usual PSHE education lessons. On the whole they felt the lessons had contributed to their knowledge and understanding of the main themes of the programme and helped improve skills such as handling pressure and making decisions for myself. Pupils were more positive in schools which had engaged fully with the programme and, primary pupils in particular, were often very enthusiastic about particular lessons.

Three quarters of the schools that had piloted lessons had received some training or induction and while not everyone thought that training was essential, most teachers suggested it was very helpful, not least in terms of becoming familiar with the programme. Much of the emphasis seems to have been on accessing the digital tools rather than promoting a life skills approach.

The report recognises that part of the challenge is to improve provision for PSHE education, particularly in some schools, but makes a number of recommendations to support the effective use of In:tuition and a successful role out of the programme.

1. Steps need to be taken to increase course fidelity. It is proposed that this can be assisted by reducing the number of digital tools, identifying a variety of curriculum models and time frames and, where possible, reducing the length of the programme.
2. Emphasis should be placed on promoting In:tuition as a core life skills course rather than an alcohol resource based on a life skills approach.
3. It is important to build in opportunities to assess learning.
4. The key digital tools need to be more accessible, for example, by identifying alternatives or including them as part of an integrated teaching tool rather than having to access them individually.
5. Include pupil-orientated PowerPoint presentations to take them through the lessons.
6. Provide a ‘lesson-plan’ format for each lesson in addition to the comprehensive notes.
7. Encourage engagement with the ‘home learning’ activities, perhaps by rebranding them.
8. Develop a programme of training and ongoing support which promotes a core life skills approach including links with other themes in PSHE education and whole school developments such as healthy schools.
9. It is important to consider how best to engage the hard to reach schools, those that are less committed to PSHE education.

We are grateful to Alcohol Research UK for funding this independent evaluation. We would also like to thank the schools, teachers and young people who contributed, for their time and co-operation.