[Eval 08] Week 4

I have chosen the guidelines SD1 and SD8 as the guidelines I would like to focus on.

For a bit of background, I work with home based workers who have to do training as part of their professional development. They are independent contractors and therefore not on a salary which would cover study time. These two guidelines probably address two of the most significant issues we have currently.


SD1: Are students able to agree some or all of their learning goals in negotiation with teaching staff?

We have major issues with trainees starting then not completing each module. This is frustrating for both the trainers and the office. Trainers are paid upon completion of each trainee completing the module, however over the last year we have had a dozen or so who have started, then decided for whatever reason not to continue. This is also frustrating for the office as we train people because we have a need for a certain number of people to be proficient in certain areas, if the trainees drop out then we are left without the required number of people able to do the work.

Therefore the issue is, how to ensure that people complete their exercises within an acceptable timeframe? Currently, we do not negotiate with each person their own individual timeframe, however I think that this is something which needs to be put in place at the beginning of each module, with a record kept both by the trainer and the office. If the timeframe is not being adhered to, then some discussion and possible renegotiation with the trainee needs to be done. To do this will also give the trainees some clear indications of how much we expect from them and would form the basis of a study plan. The trainers would also benefit from knowing what to expect from the trainees and would be able to better balance their workload.

SD8 – how to assess the value the students give to the course? This is something that has not been done before, apart from the basic 1-5 rating schemes which, as they are emailed back to me, are probably less than truthful. I have thought long and hard about this since we had the computer conference 2 weeks ago. There is the basic problem of how do you do it in order to receive truthful, constructive feedback, and who do you ask? As in all areas of life, there are people who would never have a good word to say about anything, therefore if such a person provides damning criticism of the course, is it valid feedback? However, to exclude such a person from a feedback exercise, isn’t this also tipping the balance? I think that the method of feedback collection has to be done so that people cannot make sweeping criticisms and if they have something very negative, or positive, to say then it must be backed up with examples.

There is also the issue of actually getting people to complete the feedback in a timely fashion, or even to complete it at all. Private companies seeking feedback from their clients often offer incentives such as prize draws etc, unfortunately, we do not have that luxury.

I have looked at online survey sites such as Survey Monkey which anonymise the feedback, taking away the fear of speaking your mind. The site also provides different ways of obtaining feedback, rather than just a 1-5 rating scale. I definitely need to look at alternatives to the traditional way we collect feedback, which isn't working.

importance of evaluation / quality

Week 3 already - where has time gone??? Am finding that I'm already trying to do more things online - just set up a discussion forum today for a training exercise that I'm working on - that wouldn't have happened even last week!!

I have just realised that I haven't posted my thoughts on evaluation yet, so here they are:

> Why is evaluation important to you and how do you define it?

Evaluation is really important to me as without it how do we improve on what we are doing or even know that something needs to be improved. I am not the kind of person who likes to plod along doing the same thing day in, day out and I frequently evaluate how I do things and how I can do them better. I like to have a timeline on how long the evaluation process takes and the rejigging that generally follows. In my line of work, if I don't do any evaluation on my own position, the training courses I run, etc... it would never be done, therefore in order to acheive any personal satisfaction for what I do and to ensure that I'm doing the best by the trainees, evaluation is paramount. For our transcribers/trainees if they are not evaluated regularly, even the ones who have worked with us for ages, then how will they ever progress?

Defining evaluation is not so easy - but for me, as a former teacher I probably use the format, "test, teach, test" to a certain degree in all my evaluation. MORE TO FOLLOW - HAVE TO GO AND DO SOME WORK :-)

Hi there,
These are my thoughts on some of the eLearning guidelines. Look forward to reading what everyone else thinks.
Cheers
Lesley

''SD1: Are students able to agree some or all of their learning goals in negotiation with teaching staff?'''
We currently have time limits in place for all of our training modules, however I do recognise that some degree of flexibility is important. Our courses are to be done over and above each learner’s work and therefore it is important that each of us involved in the process, including the learner, understands and factors in some negotiation of goals, and their timeliness. This is something I definitely need to improve on within our structure as some of our trainers can be quite inflexible and some of our trainees, if given too much flexibility, would never complete their modules.

ST10: Does the teacher provide advice to students on netiquette: how to communicate effectively and ethically for online discussions and email?
Believe it or not, we currently do not encourage our learners to communicate with each other via email or online discussion. We have had problems with abuse of emails in the past and have actively sought to avoid repeating that. However, I am looking at hosting our modules online via moodle or the like and where there would be a supervised chat room which would be closely monitored and learners would be provided with strict guidelines in order to avoid any abuse of the system. I feel this can be a really useful support tools for distance learning and would like to be able to confidently implement it.

SD8: Is there a mechanism in place to assess how students perceive the value of the course? We have an evaluation sheet that we send out at the end of each training module. This is a very traditional form asking for a rating between 1-5 for each training module, a rating for the feedback and marking comments and asking for suggestions for improvement. These sheets are to be emailed back to me, however I find that if learners have had any frustrations during the module they will have resolved them by the end and are so pleased to have completed it they don’t give feedback on it. There is also the fact that it is not anonymous and they don’t want to seem too critical. I definitely need to look at alternative methods of course assessment.

MT2: Are assessment and examination systems in place to ensure student identity, time limits and security? We currently use a very ad-hoc system which is open to human error (mainly mine!). Our trainers tend to do everything by email – learners are emailed their training exercises, completed exercises are returned to trainer by email, marked copies are returned with feedback, then the pass/fail result is emailed to me to update my spreadsheet. As I have said before I am currently investigating using moodle which would hopefully ensure better systems and also increased security. Currently, we do not ask learners to confirm their identity, other than an initial affidavit when they first start working for us. This is something which we do need to work on, particularly given that we are looking to obtain NZQA accreditation this year.

Week 2 - off to a good start

... very impresssed having Flight of the Conchords in the presentation! Made my day! Have started thinking about why we need to evaluate our courses - a lot to think about!
Anyway, looking forward to talking to some of you on Thursday at lunchtime and hopefully getting the elluminate computer conference to work on Thursday evening.
Have a good week!
Lesley

Hello

Hi there,

Wow, this is my first blog - ever! My niece will finally think I'm "with it"! Well, I'd better introduce myself - I'm Lesley (Fulton) and I live in Christchurch. I am originally from Scotland and emigrated to NZ in 2001 - and love it! I have a Scottish husband and 2 little kiwi kids (aged 3 & 5).

I work as training and QC manager for a digital transcription company. I create training courses to be done remotely for our home based transcribers and I am currently looking at getting NZQA registration for the training courses we offer. Whilst doing all the paperwork for NZQA I realised that the way we run our courses has never been evaluated and could be doing with being closely looked at.

I am hoping to use my own work for an evaluation project as it would be a very worthwhile to help me understand and evaluate how, and if, our courses meet the learners' needs. I was an EFL teacher for many years and am used to doing needs analysis for learners, however I do find it hard in the distance learning situation to do this and as our courses are more aimed at fulfillng the needs of our clients, I think I need to look at evaluating if our courses also meet the needs of our transcribers.

I'm really looking forward to getting started with this and meeting you all.

Cheers

Lesley