Learn how to collect evidence
1. Set objectives
Before you begin to collect evidence of learning outcomes, decide
- Why you are seeking this evidence
- What kind of information you will collect
- Who will collect it
- When it will be collected
- How to use the results to improve what you do
What do you want to find out? Keep it simple and focussed
- What are the learning outcomes of a visit to an archive?
- What new knowledge did students learn during a visit to a specific exhibition and how did this knowledge increase their understanding?
- How did using the library change attitudes towards libraries and learning?
You are trying to understand
- Whether, how and what learning occurs
- What the outcome of this learning is for the people concerned
- What the broad impact of this learning might be
2. Review existing data
Review the data you collect already
- Comment books
- Teacher's evaluation sheets
- Letters from users
- CIPFA plus survey
- User surveys
- Project evaluation forms
- Other...
For each data source, ask the question: does it give evidence of learning?
- Yes
- No
- Maybe?
Reflect on the best way to collect data to generate evidence of learning
- Examine your current data sources for evidence of GLOs - you can practice this process in Test your knowledge of GLOs
- Decide whether you need to develop new research methods - there are some ideas in the Researcher's toolkit
- Or adapt some of your current research tools to generate evidence of learning
3. Develop your research tools
Selecting your research method
- Will it generate the information you need?
- Will it produce qualitative or quantative data?
- Can you measure the quality, range and diversity of learning?
- Is it simple to use and easy to administer?
- Is it comprehensible to and suitable for the target group?
- Do you need to used it in conjunction with another method?
- Timeliness - when to undertake and for how long?
- Will it generate a large enough sample?
- Who will do the research?
Different research methods will provide you with different types of data
Quantitative data
This will help you to evidence how often things occur among which segements of your audience.
Quantitative data
- Will enable you to assess how often things occur among which segment of your audience
- Is usually gathered by questionnaires or surveys with closed or multiple choice questions
- Or interviews based on a survey questionnaire where the interviewee is invited to respond to a series of closed or multiple choice questions
Qualitative data
This is a useful tool for exploring and understanding learning in depth. A typical study will involve a group of around 10-20 participants.
Qualitative data
- Will provide you with a detailed picture of what happens under which circumstances
- Requires a larger number of respondents to ensure the sample is representative
- The sample may be targeted at a specific category of user or may represent a broad cross section of users
- Is gather using tools like open ended questionnaires, comments books and focus groups
4. Your research method - Reducing bias
Using the Generic Learning Outcomes enables you to structure evidence of learning that otherwise may be labelled as unscientific or anecdotal. Take these steps to reduce bias
- Use more than one tool - Triangulation
Multiple methods used within one research project can give you data that can be compared and contrasted to give a more detailed, reliable and rounded picture. Using the same method with different types of participant can give you different perspectives on an event, activity or process
- Be systematic and structured
Use the GLOs to help you structure the learning outcomes when collecting and reporting on your evidence. Work with a colleague to see how your judgement about the GLOs compares with theirs. Over time you will begin to form a systematic way of judging the GLOs yourself
5. Your research method - A consistent approach
Be clear about what you have done to find your evidence
- How you have done it e.g. what tools you have used, users targeted
- Why you have done it this way
- Who collected the information
- The limits of your study - what you did not do and why
6. Your research method - Ethics
Consider the following issues
- Honesty - are those involved in the research clear about how and why it will be used?
- Ownership - do you need permission to use the evidence you have collected? Who will you need to get permission from - especially regarding students or children?
- Safety - have you considered the safety of the people involved including researchers as well as those you are targeting?
- Anonymity and confidentiality - can you guarantee this?
- Professional integrity - have you maintained this to the best of your ability?
7. Develop your research tools
Use the GLOs to develop specific questions that focus on learning outcomes
You can adapt or design new research tools that encourage people to describe their learning experiences in ways which you can then analyse. You have three options
- Adapt one of the research tools in the downloads section of this website for your own use
- Adapt one of your existing research tools so that the questions focus on learning outcomes. There are some examples in the research question bank (Word, 100kb)
- Or you can start from scratch designing a new tool based on the GLOs
8. Code and analyse data
Coding data
Coding helps you to use the GLOs to
- Organise your data
- Analyse this by examining and interpreting it in a robust and consistent way
- Show a transparent audit trail of data collected, sorted and analysed
- Data needs to be coded before it can be analysed and interpreted
Have a go at coding data in the test your GLO knowledge interactive quiz.
Analysing data
An example of how you might approach data analysis
- Select a sample of around 100 returns, for example comments cards
- Count how many of these respondents have provided strong evidence of learning and express this number as a percentage of the total
- Group the data into categories, depending on what you are trying to find out, e.g by audience - adult/child, or by GLO - skills, atttitudes etc
- Count the comments and calculate the outcomes of your research, for example - from the number of comments that did provide strong evidence of learning, identify the sub set that demonstrated new skills had been developed. Then you could identify how many of these were adults and how many were children, for example
- Then start to make contrasts and comparisons
There are two templates which explain how to set up an excel spreadsheet to analyse your data
9. Presenting your evidence
The meaning and significance of your results
- What do you now know about what people have learnt?
- Do you still need to find out more about a particular type of user or a particular type of learning?
- How does this change or add to the picture you already have?
- How are you going to use his evidence to help you imporve what you do?
You can use this template of a Report structure (Word, 46kb) showing the key areas/headings you should include in a report identifying the main outcomes of your programme/project.