Tag Archives: teaching

How we assess mathematics students: a workshop at BMC

If you are attending the British Mathematical Colloquium next week look out for a workshop on ‘How we assess mathematics students: a survey and case studies’. This is being run by our assessment project ‘MU-MAP – Mapping University Mathematics Assessment Practices‘.

This project was funded as a result of the HE Mathematics Curriculum Summit, which was concerned that mathematics at HE could benefit from a wider range of assessment methods but that the research wasn’t available to the community to inform assessment decisions. The project is completing a literature survey of assessment practices, developing case studies and studying the costs and effects of change in assessment methods.

The workshop details are available on the BMC 2012 website. The abstract is below:

This workshop will present findings from the MU MAP Project: Mapping University Mathematics Assessment Practices.
MU MAP (supported by the MSOR Network through the Mathematical Sciences HE Curriculum Innovation Project) surveyed assessment practices across university mathematics in the UK and developed resources in the form of case studies of assessment of mathematics at undergraduate level. In the workshop we will present results from a survey of assessment methods in UG mathematics, and invite mathematics lecturers who took part in the project to present their case studies of assessment. We will also discuss the costs and effects of the change in assessment practice in the light of the case studies presented.

Interim report: “Development and evaluation of methods aimed at individual lecturers for producing flexible and accessible learning resources to enable inclusive curriculum delivery in mathematics”

The following interim report has been submitted by Emma Cliffe and Jane White for their project looking at methods to produce flexible and accessible learning resources in mathematics.

Progress:

A literature and technology review, coupled with survey responses and some student feedback has been used to define the requirements for the methods to produce accessible mathematical learning resources.

The review of the literature provided confirmation of the formats which departments may need to provide to disabled students and some guidance as to current methods specific to producing mathematical documents. Basic test use of individual identified methods ensured we had an understanding of the current base capabilities of a variety of technologies.

A survey was produced and staff from three institutions were invited to respond. This survey aimed to capture current practise in the production of mathematical learning resources by individual staff. Respondents were additionally asked if they were willing to provide representative samples of their learning resources in the underlying production formats. The survey had 45 respondents from three departments and 16 members of staff agreed to provide representative samples. Of these, 4 staff offered resources for research purposes only and 12 staff agreed that in addition we may report anonymised quantitative data and anonymised partial or full quotations from the files provided. This collection of samples was outside the original planned work but we felt analysis of ‘live’ samples would provide a strong basis for our recommendations as well as forming rich case studies for possible inclusion in the output resources.

A request for input from disabled students in mathematics currently receiving notes prior to lectures received only one response. We were able to mitigate this by referring back to feedback on notes already in production at Bath and we intend to contact students again once we have example resources for them to trial.

Current activities:

The collected case study samples have provided a body of test inputs to the technologies we identified in the literature and technology review stage. Analysis of the provided files, the interaction of these with the identified technologies and of the technologies with each other when working with these examples is ongoing. This analysis is being used to formulate and adjust the recommended methods for producing masters which can be automatically transformed. We will also be able to report on our experiences of working with legacy documents and to refer to case studies in the outputs.

Dissemination activities:

We gave a short report on our work to date at the University of Bath HE STEM Seminar on Monday 30th January 2012.

Future activities:

The collection and analysis of representative samples was not part of the original plan of work. However, the collection allows methods to have a sound footing prior to use by a small number of staff to produce notes for current students and enables us to report on case study documents. The trial and iterative evolution of methods, which was to take place in January will now take place later in the project, be of a more limited nature and start from a stronger base.

The main member of staff working on the project was away for a period during February. In order to ensure that the project reports in May as planned additional hours of work have been assigned to the project throughout March, April and May. The creation of instructions and examples will take now take place alongside the small trial and adjustments to the methods. This will allow the instructions to evolve in a natural way as the staff and students report back on their experiences. Analysis of costs, barriers and risks, the final report and presentation of the project outcomes will take place in May as originally planned.

Invitation: Mathematics Group Work and Asperger Syndrome

The following announcement about a meeting of our working group on ‘Group work’ on 13th March in Bath is being circulated. Please pass this message along to colleagues who may be interested.

Subject: Invitation to working group meeting, 13th March: Mathematics Group Work and Asperger Syndrome

Dear All,

This project is looking at the advantages and disadvantages of group
work used in Mathematics degree programmes especially in relation to
students with Asperger’s Syndrome.

Our aim is to build a community of academics that use group work for
assessment and developing graduates’ skills. We realise that students
with Asperger’s Syndrome may have difficulties participating in group
work thus hindering them from accessing the benefits particularly in
terms of graduate / employability skills development.

Our first meeting will be held at the University of Bath on Tuesday 13th
March from 11am – 3.30pm approx. There will an opportunity for all
involved to share their thoughts and current practices. Speakers will
include Barrie Cooper (University of Exeter) on group work in
mathematics and Daniel Aherne (National Autistic Society).

Please contact Noel-Ann Bradshaw (n.bradshaw@gre.ac.uk) and Emma Cliffe
(E.H.Cliffe@bath.ac.uk) if you would like to attend the meeting at Bath
on 13th March, are interested in attending a subsequent meeting at
Birmingham or contributing to this work in any other way.

Please pass this message along to colleagues who may be interested.

Kind regards,

Emma Cliffe and Noel-Ann Bradshaw

Group Work Working Group

We are supporting a working group on group work this semester. The following information is available on this:

This project is looking at the advantages and disadvantages of group work used in Mathematics degree programmes especially in relation to students with Asperger’s Syndrome.

Our aim is to build a community of academics that use group work for assessment and developing graduates’ skills. We realise that students with Asperger’s Syndrome may have difficulties participating in group work thus hindering them from accessing the benefits particularly in terms of graduate / employability skills development.

Our first meeting will be held a the University of Bath on Tuesday 13th March from 11am – 3.30pm approx.  There will an opportunity for all involved to share their thoughts and current practices.

Please contact Noel-Ann Bradshaw (n.bradshaw@gre.ac.uk) if you are interested in attending a meeting or contributing to this work.

Workshops: Being a professional mathematician & Placements for maths undergrads

We are supporting two workshops in May at the University of Greenwich for which registration is now open.

  1. The University of Greenwich is trialling an approach to placements that sees students placed in local companies for short periods each week. We supported Tony Mann to evaluate this process and make the results available through a case study and workshop in the project Models of industrial placements. The workshop will be Placements for Mathematics undergraduates on Monday 14th May 2012.

    This workshop will explore different ways in which mathematics undergraduates are gaining employment experience through work placements as part of the curriculum.  These include traditional sandwich placement, various opportunities to undertake placements in schools, and other examples.

    Further details and registration

  2. Tony Mann, Greenwich, and Chris Good, Birmingham, are building a set of resources on working as a mathematician and the development of mathematics with guidance on how to use these in the curriculum in the project Being a professional mathematician. This project will run a workshop Being a professional mathematician on Tuesday 15th May 2012.

    The HE Mathematics Curriculum Summit identified that mathematics departments wish the curriculum to include material on the culture of working as a mathematician. The National HE STEM Programme Mathematical Sciences Curriculum Innovation Fund has supported a project to prepare teaching materials and guidance on how these can be incorporated into the curriculum. This workshop will present some of the resulting materials and discuss ways in which they can be used.

    Further details and registration

‘Alternatives to lectures’

I have been asked to speak at the third Media Enhanced Teaching and Learning (METAL) Workshop at the University of Nottingham on 11th January 2011 with the title “Alternatives to lectures”. This series of four workshops is part of a project in using media in teaching and learning at the University which my project is supporting. Here, roughly, is what I will say.

At the first METAL workshop I spoke about effectiveness of lecture capture. You can watch a video of this as Further uses of screencasting – but are they effective? or read a write-up as Lecture capture technology – technically possible, but can it be used effectively?

As part of that talk I looked into the link between use of lecture recordings and achievement. One study identified as a positive behaviour as students coming to class then using the video recording to revisit points they struggled with. On the other hand, skipping lectures to watch the videos instead seemed to be a detrimental approach.

I also considered what might be the effect of lecture capture on attendance. The studies I found seemed to indicate a split here. Traditional, non-interactive lectures where the students watched, listened and copied what the lecturer wrote on the board observed a decrease in attendance. Those lectures which included an interactive component did not observe such a decrease in attendance. The implication might be that if the video recording faithfully replicates the lecture experience then students see little point in attending.

These results, taken together, seem to suggest that increasing interactivity in lectures encourages students into the positive behaviour mode. A few things are being conflated here and it’s all based on small scale studies, but a question is raised about whether traditional lectures are really that effective. My talk tomorrow will draw on this theme to suggest methods to increase interactivity.

The direct inspiration for this topic being on the workshop schedule is an American RadioWorks documentary Don’t Lecture Me, part of a series on 21st century ‘college’ (in the American sense) education.

Part of this talks about students’ preconceived ideas about the physical world and the effect this can have on their understanding of physics, saying:

One reason it’s hard for students to learn physics is that they come into class with a very strong set of intuitive beliefs about how the physical world works… It turns out though that many of these intuitive notions do not square with what physicists have discovered about how things actually work. Most people’s intuition tells them if you drop two balls of different weights from the second story of a building, the heavier ball will reach the ground first. But it doesn’t – and this is a very difficult concept for most students to understand because they already have a concept in their mind that’s in conflict with this new concept.

Giving his students a conceptual physics test, Eric Mazur reports:

When they looked at the test that I gave to them, some students asked me, “How should I answer these questions? According to what you taught me, or according to the way I usually think about these things?” That’s when it started to dawn on me that something was really amiss.

This sort of thing isn’t just happening at the applied end of the spectrum; it can happen in pure maths too. I remember reading some work by Lara Alcock and Adrian Simpson, Ideas from Mathematics Education, which discusses students’ preconceived or intuitive ideas of mathematical concepts (“concept images”) – using examples such as functions, limits, groups – and how these are relied on by students above formal definitions, even when the two fail to coincide significantly. Among much else of interest in that book, they say:

Pre-existing concept images might override or interfere with the use of the definition, even when the latter is known.

This brings me to a video I saw a while ago by Derek Muller on the effectiveness of science videos. The part I want to focus on is when Muller studies the responses of students who watch a video passively. In the video, when what is said differs from a participant’s conceptual understanding they don’t notice, their test scores before and after the learning stay the same and they actually become more confident in their misconception.

I’m not sure YouTube has a very thorough peer-review policy and I haven’t read the original research but the idea is interesting. Don’t Lecture Me makes a similar claim about traditional lectures:

The traditional, lecture-based physics course produces little or no change in most students’ fundamental understanding of how the physical world works. Even students who can solve physics problems and pass exams leave the traditional lecture class with many of their incorrect, intuitive notions intact.

There’s a question here about how anyone becomes a physicist. The answer given in the piece is that roughly 10% of students are motivated to teach themselves. David Hestenes is quoted saying: “They essentially learn it on their own”. It may be that the best students (and future researchers) are learning in spite of the teaching, not because of it.

So if simply watching a teacher talk through correct material isn’t helping to challenge students’ misconceptions, what can be done?

Muller advocates presenting students with common misconceptions. In the video he describes an experiment in which participants are shown a video in which their misconception is presented by an actor and then challenged in a discussion with another actor. The participants reported finding the video harder to watch but their test scores increased.

In Don’t Lecture Me (and in life), Mazur advocates a method called peer instruction. In this, students are asked a multiple-choice question in class and allowed to vote on the correct answer via an audience response system. They are then asked to discuss their answer with students sitting near them. If two students’ answers differ then whoever is correct ought to be able to convince the other of this.

What is common about these methods is the use of discussion to challenge misconceptions. Muller uses actors while Mazur uses peers, but in neither case does an authority figure tell anyone the correct answer wholesale. I’d say using discussion to challenge misconceptions is clearly indicated as a potential strategy, with peer instruction the better for a lecture environment.

In Don’t Lecture Me, Mazur says peer discussion works because the peer recently shared the conceptual difficulties. He says:

That’s the irony of becoming an expert in your field. It becomes not easier to teach, it becomes harder to teach because you’re unaware of the conceptual difficulties of a beginning learner.

I expect the approach works because students are evolving their intuitive concept towards the formal version, rather than trying to memorise a second, formal definition in parallel (or in conflict) with their intuitive one. Alcock and Simpson suggest mathematicians are still using concept images to think mathematically, but that they are doing so with “sophisticated images which they can rely on to closely match the [formal] definition”.

A while ago Sally Barton and I did a study of a lecturer’s use of audience response system (electronic voting system, clickers?) questions in class. He took fifteen minutes once a fortnight to present a quiz of five questions to students, with the aim of encouraging students to keep up to date with their lecture notes. After voting on the answers, students were told the correct answer and directed to the module webpage for worked solutions.

First, we asked students to rate on a scale their approach to answering the questions from “I think carefully about the questions asked” to “I don’t think, I just choose answers at random”. The students whose answer suggested they were more engaged with the quizzes reported taking remedial action much more than those who seemed less engaged. However, the ‘more engaged’ students reported that they were able to keep up to date with lecture notes in this module and others (where quizzes weren’t used) equally well. This suggests the quizzes were not needed as an extra incentive to keep up to date for these students. The ‘less engaged’ students tended to take little remedial action, even when they had not known the answer and had simply guessed correctly, suggesting that the quizzes were not encouraging those less engaged students to interact with the teaching materials.

When they would take remedial action, the action taken most often by the ‘less engaged’ students was not to work through the problem again, check the model solution or read lecture notes, but was to discuss the problem with their friends.

We wrote this study up in the proceedings of the CETL-MSOR Conference 2010 as ‘Using an audience response system – what do the audience DO with the feedback?’ (pp. 12-22).

If we’re right, that this group of students are least likely to engage with formal teaching material but perfectly agreeable to discussion with peers, and if this result generalises, then peer instruction could have real positive consequences for these least engaged students.

Visual impairment and inclusive curricula

On Twitter I set up a script to tweet something every day from an archive of tweets. Today it chose to link to a report of a workshop I chaired on Visual impairment in maths, stats and operational research (MSOR). I got a reply thanking me for the link. I won’t say who because the account is private, but this person said this is really useful as they work supporting a blind student. The purpose of this post is to point to a few further links that may be useful.

First, I co-authored Visual impairment in MSOR, a report on a piece of research I was involved with. The report itself may interest and some of the references used may be useful to read.

Next, Accessibility in MSOR: one student’s personal experience may be interesting.

The MSOR Network, my current employer, runs a working group on disability, Accessing MSOR, which operates through a mailing list that you may wish to join. To join requires authorisation, so you should email the group chair Emma Cliffe and introduce yourself so she knows to approve you.

Last year my project supported a workshop on inclusive curricula and Emma Cliffe is currently preparing a booklet based on this workshop ‘Good Practice on Inclusive Curricula in the Mathematical Sciences‘. I will announce on this blog when this is published.

We have also supported a project Methods to produce flexible and accessible learning resources in mathematics. This aims to address an issue arising from the above activity. The description of the project is below:

A curriculum barrier for students with disabilities is the delivery of mathematical learning resources such as lecture notes, problem and solution sheets in inaccessible formats. The current practise of repeatedly re-typesetting notes in to produce particular formats is expensive in the long run. We will develop methods, instructions and examples by which a single master copy may be used to automatically produce a variety of formats. Thus all resources are updated from a master enabling departments to make proactive adjustments. The methods will be appropriate for use by individual lecturers/departments with access to a small range of mathematical/assistive technologies.