I know it sounds unlikely, but I've been doing it. Using some of the techniques I've learnt about at a couple of data visualisation courses, to explore data and then to present it meaningfully.I decided to use the evaluation of an event I ran a month or so ago as a test bed for those techniques – thankfully, the long lead time on my Board meeting meant I could take my time about how to do it, rather than needing to produce a good report the next day, or week. I knew the evaluation material would be a good mixture – a few numbers to crunch, some qualitative comments to try and get across, some textual analysis – and this is the kind of information I have to deal with most frequently, not huge tables of complex data. I gathered all my info online using Survey Monkey, dangling a £25 voucher as a prize for those who responded. This meant I got a response rate of nearly 50%, which is reasonable for an email survey; I could possibly have got a higher response rate with a paper tick sheet, but as my colleague pointed out, the quality of responses might not be so high. I asked 12 questions, most of which were choosing a single option from a selection (e.g. excellent, good, satisfactory, poor). In quite a few cases, I also invited comments, but this wasn't mandatory. I also had some free-response boxes, asking people (for example) to highlight their key learning point from the day. I spent a bit of time combing through the numbers, totting up percentages, and then settled down to consider what I wanted to get out of my report. I decided to split my A4 page into 8 blocks, in 2 columns. A stated aim of the day was to encourage networking amongst members, and to build the feeling of being part of a network, so I decided to put that in the top left of my report; using our perceptual understanding to put the most important thing in the first place people look. I put in figures about how useful attendees found the day (a kind of summing up), percentages of those who felt their network feeling had increased, and the positive impact this would have on their work; and then added some of the summary comments (balancing predominantly positive with a few gripes). Underneath this was a text box, quoting suggestions for future events. There were a number of similarities amongst some of these, so they were grouped for impact. The top right quarter of my sheet contains some of the textual analysis and qualitative prodding I did. A small spider diagram shows the key learning points identified by colleagues, grouped under 3 headings, with a wordle of the terms used underneath. The wordle is a bit diffuse but I like the generally positive message it gave and it works quite well as a 'flavour'. The bottom half of the sheet contains more number crunching. The bottom right quarter focuses on the single afternoon session, which was an interactive workshop, starting with a 100% stacked bar chart showing how participants rated its usefulness. There were quite a few comments on this session, and it looked at bigger generic issues of the policy and research process, so I've captured a lot of direct quotes in this quarter, including a couple that ought to raise questions for those who'll be looking at this. To the left of this are 100% stacked bar charts of the usefulness of the two morning speakers, along with key quotes. These sessions were more subject specific, so responses to these questions varied on the role and interests of the respondent. Finally, I cover the practicalities – venue and catering ratings – in two pie charts in the bottom left corner. They are tucked away but easily indicative of the basic questions that post-event evaluations always ask. The best thing about writing this post has been realising that the single side of A4 much more effectively and succintly conveys the information I've been trying to get across in a blog post, and that, in those terms at least, my experiment has been a success. Once it's complete, I can post it up here so you can see how I've got the information from the raw data, and judge for yourself how successful I've been. (tag: data visualisation, data, information, research)
OK, this sounds good. If you have scan technology to hand, perhaps you could put up an A4-sketch for the time being. Always love seeing the early ideas.
Also, highlights the importance of getting your aims in sharp focus from the start; so important for getting good results. Sounds obvious, but often overlooked.
If you’ve done some of this in Excel, perhaps we could start a ‘cool things to do in Excel’ list (for when you don’t have a 100k ESRC grant to hand) 🙂
You, Warren, can probably have privileged access to the actual PDF as it develops. And I do wonder whether there is some mileage in showing a) how differently you can approach getting something meaningful from evaluation, and b) how obviously it is based on some other examples seen at data visualisation courses…!
The Excel stuff has been somewhere between very basic and pretty basic… i was just overjoyed to discover I didn’t have to use default colours, and to establish how much you can format in Excel if you really want to. Having an Excel-demon on hand definitely helps! But as always, having the time and space to investigate is what matters.
Oop, and one last thing… what has taken the time is the thinking – working out the aims and what the data tells us. Actually putting the thing together is relatively speedy by comparison!
Time for an update?