brown(e)learning

The weather has finally turned here in Western NY, so I’m biking to work more. I finally setup my Macbook so I can access its desktop from my home Windows PCs. Of course, I need this connection to be encrypted, so I had to tunnel it over SSH (which means using Vine Server rather than Apple’s Screen Sharing). There are numerous tutorials for doing this, but this one held to more important information: the exact PuTTY tunnel configuration.

So, more for my records than for your information, here are my screenshots.

I participated in a national web conference on assessment sponsored by a respected organization. Many of the talks were good, one or two made some great points, but one in particular had some serious flaws. It’s generally my policy not to call people out, and my purpose here is not to critique the speaker, so I’m going to do my best not to cite him/her.

When this person presented his/her definition of “Assessment,” a participant sitting next to me asked me what I thought of it. “It’s half-right, but gets too much into evaluation,” was my response. As the presentation progressed, I noticed that the line between assessment and evaluation become so blurred that the speaker was using the terms interchangeably.

When the presentation was over, there was an opportunity for us to ask in questions via a chat form. Mine was first in line: “What’s the difference between assessment and evaluation?” The speaker hummed an ha’ed, claimed that this was an open question that “we’re still working on,” and stated that he/she used the terms loosely.

No, no, no. It’s only an open question to people who are trying to figure it out for themselves rather than leveraging the mountains of theoretical and empirical work that several professional organizations have already completed. There are still various definitions for each term, but here are two that draw a distinct difference:

Assessment is the systematic process of gathering data to inform decisions (adapted from Nitko & Brookhart, 2005).

Evaluation is a judgment of quality, merit, or worth.

Obviously evaluative judgments can be used as assessment data, and assessment data can inform evaluation judgments. But the assumptions, theories, and methods are tailored to their respective end goals (judgments or data).

As these fields have developed, assessment has become a bastion of positivist assumptions and quantitative methods, while evaluation was a safe harbor for relativism and qualitative methods almost from its inception. Perhaps because of this bifurcation (and the persistent classification of qualitative methods as non-scientific), colleges and universities have offices of “assessment,” that really perform evaluation.

In common speech, there is little harm in not distinguishing between assessment and evaluation. But when we want to get our hands dirty and accomplish something with these processes, and especially if we’re elevated to the point of lecturing others on how to conduct this work, we ought to have the distinction clear in our mind.

A few months ago, Google created a honeypot trap for their competitor. While that’s not quite what I’m doing, i would be wrong not to recognize Google’s efforts as an inspiration in this.

I migrated brown(e)learning to a hosting company’s server last month. Buried somewhere in the sign-up form was an offer for an additional service that supposedly would scan the site for vulnerabilities. If none are found, the firm behind the service would allow me to display their badge on my site. I didn’t want the service, didn’t even notice that I had signed up for it, but now I’m getting semi-daily emails letting me know 1) there are no vulnerabilities on my site, 2) how I can complete the registration process, and 3) how to display their badge on my site.

So, I’ve had enough. I created a page that contains an obvious SQL injection vulnerability. I’ll link to it from several of the pages on my site to make sure the scanning service can see it, and then determine the legitimacy of the service based on whether they notice it.

BTW, the MySQL user for that page only has SELECT privileges to an otherwise unused database, so you can’t do this…

Oh, and that page can be found at: http://brownelearning.org/test/vulnerable.php…

This week’s topic in Understanding Educational Research was ethics. I always enjoy such topics that, despite being infinitely open-ended, have been reduced to a set of guidelines that are – at their origin – completely arbitrary. As I muddled through readings on traditional, contemporary, Kantian, Foucautian, moral, methodological, and situational ethics, I envisioned a young, well coiffed Tom Cruise screaming a post-modern critique at Stanley Milgram (played by Jack Nicholson ).

Cruise: I want the (socially-constructed) REALITY!

Nicholson: You can’t handle reality!

Son (or Daughter), we live in a world of science. And that science must be carried out by people with data.

Who’s gonna do it? You?

We, scientists, have a greater responsibility than you can possible fathom.

You weep for hurt participants and you curse quantitative methods because you have that luxury.

You have the luxury of not knowing what I know: That harming those participants, while unfortunate, yielded data.

And our analyses, while demeaning and tautological to you, brings good from those data.

You don’t want reality.

Because deep down in places you don’t talk about in your autoethnographies, you want our data.

You need our data.

We use words like rigor, significance, validity… we use these words as the pathway to scientific progress.

You use ‘em as a punchline.

I have neither the time nor the inclination to explain our methods to someone who is fed, clothed, transported, and even kept alive by the very science we carry out and then questions the manner in which we carry it out!

I’d rather you just said thank you and wrote your critical incident papers.

Otherwise, I suggest you gather your own data and do some real research!

Either way, I don’t give a damn how you think science should be done.

We performed this as a readers’ theater in my grad class with interesting results. Eighty percent of the my students in that section were science education grads, and, while they would agree with Nicholson’s lines, they understood that he was the villain in A Few Good Men.

I’ve taken on some added responsibility this semester. Specifically, a good friend in our assessment office has needed some time off, so I’ve shouldered a small part of her burden.

One task she normally takes care of is to create stacked bar charts of our NCATE-accredited programs’ assessment data. I was asked to complete this task before a meeting this week, but I found the process more than tedious. I was so frustrated by the number of steps required to render each chart, that I exported the entire dataset (dating back to 2005) and wrote-up a simple web form to do it all in one step.

When I was done I sent a link to my colleague who usually creates the charts. Here was her reply:

JEREMY!!  HOW COOL!!!!  Can’t tell you the number of people I have asked through the years for the automatic chart and table generator!!

(And then she quickly made a plethora of feature requests.)

Some code for money, others code as a form of expression. I wonder though, how many of us, deep down, like to code to help people? Is altruism a motivation for coding?

I’m part of a federal grant looking into an aspect of general education at two campuses in Western New York. (I’ll spare you the boring details.) Suffice it to say that I’ve developed a rubric for a certain aspect of critical thinking, and I’m reviewing a random sample of student work to see how much of this aspect is present. It should be noted that although the sample of students from each course is random, the assignment and course are not randomly sampled. Rather, each instructor either created an assignment or selected a pre-existing assignment that would exemplify this aspect of critical thinking.

As I’ve read through these papers and projects one thing has struck me: Regardless of the presence or absence of critical thinking, there is an incredible lack of diversity in thought.

It reminded me of when I was in graduate school and one of my professors farmed us out as consultants to other professors. (At the time I thought he was giving us practical experience; now I realize he was slapping us in the face with reality.) One professor from a social science department told us that his objectives included “Students will interpret ABC in light of XYZ.”

“That’s great,” I responded. “And how many correct interpretations are there?”

He didn’t hesitate, “One.”

We have a well-established word for only one right answer: indoctrination. Though it has a pejorative connotation, indoctrination is not always a bad thing. But it should never be confused with critical thinking. Critical thinking is not when your students think like you, it’s when they question how you think.

Michael Johnson used this CommonCraft clip in his Social Media course this week.

The analogy is a bit stretched, but the point that social media allows producers to communicate with their consumers is a good one.

But then Slashdot linked to this post by Jeff Vogel, an independent game developer. He argues that creative people should not participate in social media around their work. Such interactions are unproductive at best (too much noise for any signal) and counterproductive at worst.

So how do we bring these two perspectives into some sort of harmony?

It’s important to note that CommonCraft is looking at production (for consumption) while Vogel considers creation alone. The two are not synonymous; there are plenty of people (even in the gaming industry) who make things just to make things. They don’t necessarily care who appreciates or doesn’t appreciate their work. The “cacophony” of social media is less useful to these artists.

I just signed off on my ninth thesis since Thanksgiving, so the blog and my personal reading have suffered a little bit. But, just so I have this written somewhere, here are some of the books I’m trying to finish and/or want to read:

The Colossal Book of Mathematics by Martin Gardner
The Brain that Changes Itself by Norman Doidge
Black Swan and Procrustean Bed by Nassim Taleb (after reading the Q&A with the author in Time)
All the stats books in Andrew Gelman’s reading list
Tensions in Teacher Preparation: Accountability, Assessment, and Accreditation by Nancy Wentworth and Lynnette Erickson

Michael ponders the motivation for expanding one’s learning network and raises some good questions. He seems to shake the Randian objectivist theory that people follow (Twitter, etc.) others only if doing so improves the follower’s position. The objectivist view needs to be challenged, but Michael’s post is an opportunity for me to add my two cents…

My biggest concern of the Internet’s tribalization is that people may tend to follow those who agree with them. While it is possible to learn from like-minded individuals, I believe the greatest intellectual progress derives from the mutual exposure of conflicting positions.

My second-biggest concern is with multicast status/location/updates/tweets/buzzes, etc. When someone uses a forwarding service to update several social networks at once, it becomes a fire-and-forget activity; they would have to check several sites for responses to their update, so there is less of a chance that a meaningful (even brief) conversation could result. This wastes some of the great social learning opportunities.

Like Michael, I don’t have a solution to these problems. I’m just throwing them out there.

Back in the late 1990′s, a new administration at a large private university asked departments to declare their “final utility.” The French department, in which I was studying, mocked that request: If you don’t understand the importance of the Humanities, then we can’t explain it to you.

Fast forward a decade and one French professor, now the associate dean, writes this for the Chronicle of Higher Education:

Everybody, including me, is certain that one of the biggest problems we in the humanities confront is fiscal. But even more than that, we have a language problem: We don’t know how to articulate our value in terms of the accounting metrics and the models of utility that govern top-level allocation decisions. Few people, especially among the new styled MBA administrators, understand the humanities, and if they make the effort, they struggle to see the relevance or value of them. Who among us has not experienced the torment of trying to articulate our existence to people for whom the relevance of a college degree is limited to its immediate market value? Add to this the new context of global economic crisis and high unemployment and our self-justifications seem strained, if not entirely implausible.