brown(e)learning

There was a web meeting today on instructional design. I was in another meeting, so I was unable to tune in, but I was interested in something from the session’s abstract:

Author and designer Victor Papanek (1985) defines design as “the conscious and intuitive effort to impose meaningful order” (p.4) This can mean that whether we are involved in instructional design, or in other endeavors in which something is to be created different than what existed before; “all that we do, almost all the time, is design, for design is basic to all human activity” (Papanek, 1985, pg. 4).

Beginning with Papanek’s specific definition, I can’t get over the union of “conscious and intuitive effort.” My dictionary places these adjectives in slight opposition: Conscious entails awareness, while intuition occurs in the absence of reason. Perhaps these terms were included as either-or descriptors (e.g., “both conscious and intuitive efforts may be counted as design”). Such inclusion would be consistent with another point I’ll make below.

I’m less certain the requirement that design assumes the intention to “impose meaningful order.” That limitation appears useful to distinguish design from its superset art, but I waiver on ostracizing creation without meaning. I’d like to ponder the implications of this specification a bit longer.

I noticed that fields attempting to define themselves often do so with implicit goals of inclusion. This motivation drives the theorists toward definitions that are no less abstract than the term’s generally accepted meanings. Take, for example, Papanek’s comment that design is “all that we do, almost all the time.” To quote David Merrill’s protestations against learning objects, “If something is everything, then it is nothing.” In other words, defining something to encompass anything or almost everything renders the term useless; it would be more useful to define what design isn’t.

Perhaps the difference between Papanek’s definitions and my own efforts to lock-down the meaning of design has been the angle of attack: Papanek was a designer who believes his field to be truly broad, while I was a design student trying to delineate the difference between my field and art. I eventually accepted this distinction:

Design is creation within constraints.
Art is creation that may or may not be constrained.

By my definitions, some design is clearly a subset of art, but art is a larger umbrella. I placed the “constraints” in design because I noticed that in practical fields – from my father’s engineering, to my graphic design (and later to my software, research, and instructional design) – the design process included acknowledgement of the functional requirements and limitations of the work.

I’m not sure that either of these definitions is adequate, but hearing Papanek’s definition stirred the memories of a long-settled self-debate.

I attended a regional conference on instruction and technology this week. I’ll admit I didn’t know what to expect – I had never been to anything smaller than a national conference – but I should have been able to predict that the sessions would be highly variable in their quality. Of course, there were the vendor demonstrations and the how-to reports devoid of data, but there were also more discussion-oriented presentations on issues of common-yet-unexplored techniques, and business reports on local issues.

I was satisfied with the quality of several of the presentations I attended, especially one on rubrics wherein, once the presenter discovered (by – gasp! – asking the audience questions) that most of the attendees had experience developing rubrics, she decided to let us do the talking. She would pose a question (which she knew to be the topic of her next slide), and then she would moderate our conversation. After four or five comments, she would flip to the next slide and show us that we had covered everything on it. Her effort in organizing the information wasn’t wasted just because the group in the room already knew it; see, we only knew it collectively, and she helped move that information from collective to individual.

Sadly, the worst hour I spent was during on keynote address. I’ve known for a while that the educational technology field is driven more by marketing hype (and the artificial demand it creates) than by the effectiveness of the technology. This presentation made me wonder the degree to which historical revisionism has helped maintain that unhealthy producer-consumer relationship.

The presenter stated as fact that the early computer-assisted educational programs translated the production mentality of early computers into a one-size-fits-all approach to education. While there were mass-produced products of this type, it’s wrong to ignore the more instructionally-appropriate efforts that included student tracking and automatically adjusted to their needs.

Ten years ago I sat on a committee with some very bright professors in the field. We discussed requirements for instructional authoring systems and, as each “cutting edge” feature was brought up, someone would ask a distinguished professor, “Didn’t [insert thirty-year-old system] have that?” He would nod or give a simple, “yep” in response.

The ed tech field’s collective ignorance of the depth and meaningfulness of their roots is not only wrong because it slights the visionaries of decades past; it’s also wrong because it allows us to be wooed by technologies that appear new, but are only new in that industry has never pushed them before.

This ignorance is most observable in our field’s unabashed embrace of the mobile “app” model. Those of us who have given decades of effort to establishing open, standards-based technology thoroughfares should be revolted that making our wares available on monopolized toll-roads is now seen as a requirement. (For more reading, see The App is Crap.)

The weather has finally turned here in Western NY, so I’m biking to work more. I finally setup my Macbook so I can access its desktop from my home Windows PCs. Of course, I need this connection to be encrypted, so I had to tunnel it over SSH (which means using Vine Server rather than Apple’s Screen Sharing). There are numerous tutorials for doing this, but this one held to more important information: the exact PuTTY tunnel configuration.

So, more for my records than for your information, here are my screenshots.

I participated in a national web conference on assessment sponsored by a respected organization. Many of the talks were good, one or two made some great points, but one in particular had some serious flaws. It’s generally my policy not to call people out, and my purpose here is not to critique the speaker, so I’m going to do my best not to cite him/her.

When this person presented his/her definition of “Assessment,” a participant sitting next to me asked me what I thought of it. “It’s half-right, but gets too much into evaluation,” was my response. As the presentation progressed, I noticed that the line between assessment and evaluation become so blurred that the speaker was using the terms interchangeably.

When the presentation was over, there was an opportunity for us to ask in questions via a chat form. Mine was first in line: “What’s the difference between assessment and evaluation?” The speaker hummed an ha’ed, claimed that this was an open question that “we’re still working on,” and stated that he/she used the terms loosely.

No, no, no. It’s only an open question to people who are trying to figure it out for themselves rather than leveraging the mountains of theoretical and empirical work that several professional organizations have already completed. There are still various definitions for each term, but here are two that draw a distinct difference:

Assessment is the systematic process of gathering data to inform decisions (adapted from Nitko & Brookhart, 2005).

Evaluation is a judgment of quality, merit, or worth.

Obviously evaluative judgments can be used as assessment data, and assessment data can inform evaluation judgments. But the assumptions, theories, and methods are tailored to their respective end goals (judgments or data).

As these fields have developed, assessment has become a bastion of positivist assumptions and quantitative methods, while evaluation was a safe harbor for relativism and qualitative methods almost from its inception. Perhaps because of this bifurcation (and the persistent classification of qualitative methods as non-scientific), colleges and universities have offices of “assessment,” that really perform evaluation.

In common speech, there is little harm in not distinguishing between assessment and evaluation. But when we want to get our hands dirty and accomplish something with these processes, and especially if we’re elevated to the point of lecturing others on how to conduct this work, we ought to have the distinction clear in our mind.

A few months ago, Google created a honeypot trap for their competitor. While that’s not quite what I’m doing, i would be wrong not to recognize Google’s efforts as an inspiration in this.

I migrated brown(e)learning to a hosting company’s server last month. Buried somewhere in the sign-up form was an offer for an additional service that supposedly would scan the site for vulnerabilities. If none are found, the firm behind the service would allow me to display their badge on my site. I didn’t want the service, didn’t even notice that I had signed up for it, but now I’m getting semi-daily emails letting me know 1) there are no vulnerabilities on my site, 2) how I can complete the registration process, and 3) how to display their badge on my site.

So, I’ve had enough. I created a page that contains an obvious SQL injection vulnerability. I’ll link to it from several of the pages on my site to make sure the scanning service can see it, and then determine the legitimacy of the service based on whether they notice it.

BTW, the MySQL user for that page only has SELECT privileges to an otherwise unused database, so you can’t do this…

Oh, and that page can be found at: http://brownelearning.org/test/vulnerable.php…

This week’s topic in Understanding Educational Research was ethics. I always enjoy such topics that, despite being infinitely open-ended, have been reduced to a set of guidelines that are – at their origin – completely arbitrary. As I muddled through readings on traditional, contemporary, Kantian, Foucautian, moral, methodological, and situational ethics, I envisioned a young, well coiffed Tom Cruise screaming a post-modern critique at Stanley Milgram (played by Jack Nicholson ).

Cruise: I want the (socially-constructed) REALITY!

Nicholson: You can’t handle reality!

Son (or Daughter), we live in a world of science. And that science must be carried out by people with data.

Who’s gonna do it? You?

We, scientists, have a greater responsibility than you can possible fathom.

You weep for hurt participants and you curse quantitative methods because you have that luxury.

You have the luxury of not knowing what I know: That harming those participants, while unfortunate, yielded data.

And our analyses, while demeaning and tautological to you, brings good from those data.

You don’t want reality.

Because deep down in places you don’t talk about in your autoethnographies, you want our data.

You need our data.

We use words like rigor, significance, validity… we use these words as the pathway to scientific progress.

You use ‘em as a punchline.

I have neither the time nor the inclination to explain our methods to someone who is fed, clothed, transported, and even kept alive by the very science we carry out and then questions the manner in which we carry it out!

I’d rather you just said thank you and wrote your critical incident papers.

Otherwise, I suggest you gather your own data and do some real research!

Either way, I don’t give a damn how you think science should be done.

We performed this as a readers’ theater in my grad class with interesting results. Eighty percent of the my students in that section were science education grads, and, while they would agree with Nicholson’s lines, they understood that he was the villain in A Few Good Men.

I’ve taken on some added responsibility this semester. Specifically, a good friend in our assessment office has needed some time off, so I’ve shouldered a small part of her burden.

One task she normally takes care of is to create stacked bar charts of our NCATE-accredited programs’ assessment data. I was asked to complete this task before a meeting this week, but I found the process more than tedious. I was so frustrated by the number of steps required to render each chart, that I exported the entire dataset (dating back to 2005) and wrote-up a simple web form to do it all in one step.

When I was done I sent a link to my colleague who usually creates the charts. Here was her reply:

JEREMY!!  HOW COOL!!!!  Can’t tell you the number of people I have asked through the years for the automatic chart and table generator!!

(And then she quickly made a plethora of feature requests.)

Some code for money, others code as a form of expression. I wonder though, how many of us, deep down, like to code to help people? Is altruism a motivation for coding?

I’m part of a federal grant looking into an aspect of general education at two campuses in Western New York. (I’ll spare you the boring details.) Suffice it to say that I’ve developed a rubric for a certain aspect of critical thinking, and I’m reviewing a random sample of student work to see how much of this aspect is present. It should be noted that although the sample of students from each course is random, the assignment and course are not randomly sampled. Rather, each instructor either created an assignment or selected a pre-existing assignment that would exemplify this aspect of critical thinking.

As I’ve read through these papers and projects one thing has struck me: Regardless of the presence or absence of critical thinking, there is an incredible lack of diversity in thought.

It reminded me of when I was in graduate school and one of my professors farmed us out as consultants to other professors. (At the time I thought he was giving us practical experience; now I realize he was slapping us in the face with reality.) One professor from a social science department told us that his objectives included “Students will interpret ABC in light of XYZ.”

“That’s great,” I responded. “And how many correct interpretations are there?”

He didn’t hesitate, “One.”

We have a well-established word for only one right answer: indoctrination. Though it has a pejorative connotation, indoctrination is not always a bad thing. But it should never be confused with critical thinking. Critical thinking is not when your students think like you, it’s when they question how you think.

Michael Johnson used this CommonCraft clip in his Social Media course this week.

The analogy is a bit stretched, but the point that social media allows producers to communicate with their consumers is a good one.

But then Slashdot linked to this post by Jeff Vogel, an independent game developer. He argues that creative people should not participate in social media around their work. Such interactions are unproductive at best (too much noise for any signal) and counterproductive at worst.

So how do we bring these two perspectives into some sort of harmony?

It’s important to note that CommonCraft is looking at production (for consumption) while Vogel considers creation alone. The two are not synonymous; there are plenty of people (even in the gaming industry) who make things just to make things. They don’t necessarily care who appreciates or doesn’t appreciate their work. The “cacophony” of social media is less useful to these artists.

I just signed off on my ninth thesis since Thanksgiving, so the blog and my personal reading have suffered a little bit. But, just so I have this written somewhere, here are some of the books I’m trying to finish and/or want to read:

The Colossal Book of Mathematics by Martin Gardner
The Brain that Changes Itself by Norman Doidge
Black Swan and Procrustean Bed by Nassim Taleb (after reading the Q&A with the author in Time)
All the stats books in Andrew Gelman’s reading list
Tensions in Teacher Preparation: Accountability, Assessment, and Accreditation by Nancy Wentworth and Lynnette Erickson