“...one value I have embraced is designing ways participation and outcomes in my MOOC can’t be measured. I don’t know what will make my MOOC successful - but I think I’ll recognize it if I see it.”
As it happens, I agree with him — partly. In the large-scale projects I have bid for and managed, there have been two forces operating.
Project planning, by Terry Freedman
One is the need to be accountable, especially if you are bidding for money, time, people’s time or other resources. (I was going to write that nobody would plough money into a project without having any success criteria in place, but then I remembered that that apparently happened with a department of education project a few decades ago. So when it came to evaluating whether or not it had been a success, they had a bit of a challenge on their hands. I don’t recall the full details.)
Thus you have to come up with various criteria that can, in theory, be measured and which, also in theory, actually matter. The main problem with this kind of thing is that you can end up being so focused on measurable criteria that you miss really important stuff — because you weren’t looking for it. (Cf the famous video of a basketball game in which someone in a gorilla suit gets in on the act and nobody notices him!)
Therefore my approach was to jump through the hoops to get the funding, and then make sure whatever we promised to improve or achieved was monitored, but at the same time have a sort of “shadow” goal.
This “shadow” goal was along the lines of saying to everyone involved: keep your eyes and ears open for great and possibly unexpected outcomes. In my experience, anecdotal evidence adds a layer of meaning to the dry statistics.
A couple of examples. Some years ago I bid for government funding for a large-scale multimedia project involving six primary schools and one secondary school, and oversaw the project when the bid was successful. One outcome we hadn’t set as a goal was that a Year 5 boy (ten year-old) who was autistic and never spoke in class, became the go-to person for video editing, explaining to other kids how to do various things.
A profoundly deaf girl gave a pretend weather forecast using sign language and a green screen.
There were lots of other stories like this, none of which of course were pre-stated goals. Therefore I am in agreement with Downes on this. You can’t know in advance whether a project will be a success, or what form that success will take, but you ought to be able to recognise it when you see it.
But you still need to satisfy the bean-counters.
