brownelearning

The TEDxNYED videos are now online, and I’ve decided to rewatch the presentations I found interesting and write up my reactions. I’ve already said (as have others) that I was disappointed in the overall experience, but that was not the fault of speakers. So, here’s a good one: Mike Wesch’s Cautionary Tale

Why did I enjoy Mike’s speech? Mainly because he brought in external material of which most of the attendees had been ignorant, and then used that new information to shed light on our field. (This is how the TED format could work for thematic conferences.) Specifically, Mike showed how “new media” can have harmful side effects. The point needs to be considered carefully by the edtech community.

I do not think the rest of this speech was as strong and, worse, demonstrated some glaring inconsistencies in Mike’s thinking – thinking that has so great potential.

For example, in one case “new media” were bad when they altered the established native culture, but then, in another case, “new media” were good because they could alter an established culture. Without explicitly establishing the criteria by which we can judge the merits of the culture (is it worth saving or does it deserve to be destroyed), the arguments canceled out.

Further, Mike failed to recognize any value (or even the existence) of the cultures “new media” create. “[New media] will be what we make of it,” he said. Well, I say new media are without our interference and we do not have a responsibility to “make” something of them.

Finally, Mike barely implied that there were limits to using new media in class. He stated that his students could change their “research schedule” (not a “syllabus”) when and how they saw fit, but they could not change the class goal. Oh, but who decided on that goal? The students? or the teacher? He did’t say. Also, if you noted the byline on the documentary his class produced, it only listed the professor’s name. I’m not saying these limits on student-centeredness are unreasonable, only that we should talk as much about them as we do the hype they counteract.

Again, this was one of my favorite speeches at TEDxNYED. I hope it starts some good conversations.

I have been satisfied with the level of acceptance open source products have earned during the last decade. But I fear that its progress may be impeded by the continued confusion between “free” and “open.” This is not a new argument, but the premises have shifted slightly. Basically, open source advocates must constantly remind the public that the YouTube, Twitter, Facebook, etc. are NOT open source.

I sat in a meeting last week listening to faculty and staff debate the future of our campus’ learning management system. Our provider has been purchased by Blackboard, which gives us four years to either migrate to the Beast, or find another way. If you know me, you know what my opinion is.

I’ve been pushing Jon Mott’s model: Let’s focus on the core functionality of what the institution must do, while supporting faculty and students’ use of non-campus (“cloud”) tools. This maintains the security over student data and restricted material without sacrificing the advantage of the emerging technologies. As we were discussing how the college IT staff could support such a model, one person spoke up:

Several faculty have started using open source sites, and they’re starting to realize that when the system goes away, they’ll lose their work.

Anyone in open source, or even with a cursory knowledge of its pros and cons would have done a double take. I think my brow furrowed into the ceiling panels. They continued.

When sites like YouTube make changes, or just close-up shop, a lot of the work you’ve put into building your content is just gone.

I usually hold my tongue in meetings, especially when deans or higher-ups are in attendance, but this was one misconception I dearly wanted to nip in the bud. Before the moderator had a chance to call on anyone, I corrected:

Just a correction, if I may? YouTube is NOT open source. It’s free to use, but the very nature of open source would actually mitigate the risks you bring up.

I spoke with the person after the meeting to give them a quick rundown on what open source is, is not, its advantages and disadvantages, and why it was so important they I correct them during the meeting.

Today I received my subcommittee’s report on the issues we’ve researched. My view is well represented in the report, but our subcommittee chair made a similar mistake. She equated cloud computing with the use of open source tools. I sent a polite correction, which was applied to the report.

Most of my involvement in open source has assumed that when the products were good enough, they would break into the mainstream. I felt that the movement had succeeded when campus IT guys installed Firefox on the lab computers. But it appears I missed the next glass ceiling: Overcoming mid-level decision-makers’ misconceptions.

David offers a parody of the future of education. Now, the hard part about responding to a parody is that you never know which components are meant to be serious. From what I know of David, I think I can figure it out, but I offer the caveat that I may be wrong.

He offers two good reasons for which the future of education will look a lot like its past:

1. Employers will continue to want a third-party validation of their potential employees’ qualifications (degrees from accredited institutions). [I think he believes this one.]

2. The societal-shifting advances of the past (academies, books, printing, etc.) failed to disrupt the traditional lecture format of education. [I think this one is facetious.]

I want to explore the second point a little deeper, even it wasn’t meant to be examined..

If lectures were as soul-destroying as many educational reformists would have us believe, then no one would use them. (Even TED would change its format.) But lectures give something that other formats of instruction do not: efficiency.

It’s akin to why so many tests – even those created by good teachers – continue to employ multiple-choice items.

But surely we can find instructional media that are more efficient than lectures. A text posted online, for example, has the potential to serve orders of magnitude more learners than the thrice-weekly professorial song-and-dance in the lecture hall. And its distribution cost would be infinitesimal relative to the cost of a live lecture. If efficiency is accomplishment over cost, then online texts are much more efficient than live lectures.

That’s where David’s first point comes into play. Live lectures preserve the gate-keeping status of the institution. The lecture hall has walls that limit participation; they only include those meeting the requirements of the institution. But it doesn’t have to be this way.

If you look at the truly innovative institutions (e.g. WGU), they have shifted the gate-keeping mechanism from credit-for-participation-in-information-distribution (the role of lecture and the lecture-hall walls) to credit-for-evidence-of-proficiency (assessments). This model affords greater efficiency in instruction without sacrificing the gate-keeping (read: profit-generating) responsibilities of the institution.

Contrary to David satirical vision, I believe that education will look different in the future. And institutions who shift from information-distribution to candidate-evaluation will own the future of education.

That headline is hyperbole in honor of Jeff Jarvis.

I haven’t written my review of TEDxNYED yet because I’m waiting for the videos to appear online. I’d like to verify my notes and memories of specific speeches before I make my reactions public. As it stands, it may take several posts to summarize my thoughts on the experience.

Many participants and speakers posted their comments immediately following the event, and, surprising to some, not all of the reviews are “parades of rainbow sparkle ponies.” While some speeches appear universally appreciated, many others left general senses of So what? and Why?

This mixed reaction led to a little back-and-forth, with TEDxNYED fans suggesting that the detractors were looking for more than the conference aspired to be. The fans point out that TED’s motto is “Ideas Worth Sharing,” so looking for anything more than that is foolish. At least one participant realized this halfway through the day, and enjoyed it much more thereafter.

But I would argue that the TED-like format of the presentations, the application-only attendance policy, and the relatively narrow topic doomed TEDxNYED from the get-go. In other words, the effectiveness of TED doesn’t generalize to smaller thematic events.

In hindsight it seems painfully obvious: Pick a random person working in educational reform or technology. Ask them to explain their work in 18 minutes or less. I’d venture from personal experience that half of them will spend the entire time justifying their work. That is, they will do nothing but explain the problems they see.

Now put those technologists and reformists in front of people involved and/or interested in those fields. Most of the audience will already be aware of the issues in education. So many of us at TEDxNYED just sat there mute, listening to vociferous preaching and approving grunts from the choir. From a particularly perspicacious commentary:

[I wondered] how much further does this get us?…and wondering, feeling the discomfort of the lack of diversity in the room, lack of real diversity in the opinions, the fear of spending yet another day in the echo chamber

I had hoped that TEDxNYED would be more than that because it was thematic. Instead it was an attempt to force the square peg of the TED format into the round whole of educational innovation.

Over at Slate, Sara Mosle has a review of Diane Ravitch’s The Death and Life of the Great American School System.

It’s filled with interesting tidbits, but I don’t know if the book would be worth reading when…

The dirty dark secret of NCLB is that we may know how to identify the worst performing schools, but no one (yet) knows how to turn them around in any consistent and reliable way. And I mean no one. Not the Gates Foundation to date. Not most charter programs. No one.

This is what I’ve been saying for years, and I blogged over a year ago. We don’t know, but there is a way to find out. Let’s tie funding to reporting requirements, but remove the performance requirements so districts are free to experiment. Their experimentation is more likely to yield workable solutions than policymaker or reformist pontification.

A few posts ago I noted that there is no Web 2.0. As Sir TB-L stated, user-to-user communication was the original intent of the web, and many us have been using it for that purpose since its inception. The only substantive difference between the web circa 1994 and Web 2.0 is that corporations have realized there is money to be made in providing a forum for user-created content. I see some strong parallels between Web 2.0 and the “universities will be irrelevant,” “they must adapt or die” chorus.

Those who insist that universities reform to embrace the Internet-age (iReformers?) assume that higher education is not already so doing. Many of us – especially, but not uniquely, in education – have long abandoned lecturing; we have shifted from memorizing facts to reasoning with them; and we have long embraced technology as learning tool. So forgive our confusion when someone says we must change to be exactly as we are, especially when we see fewer advantages and a multiplicity of disadvantages in institutionalizing our best practices.

PS – I’ll also restate this retort to those who think Higher Ed will become irrelevant: Universities existed long before they were relevant and were, in many ways, superior to what they are now. How is changing to maintain their new-found relevance a good thing?

—–

Social science in general and educational research in particular are packed with ambiguous denotations. Every time someone takes a new view on anything, they feel an urge to re-use terms from older theories; and this has left us in a disorienting clutter of nomenclature that is hardly productive. I heard two professors argue through a lunch, only to conclude that they held different meanings of the adjective “esoteric.” Does “critical” describe an issue’s importance or its research method? “Punishment,” “intelligence,” and, yes, even “validity” don’t just hold different meanings in education than in real life; they hold many distinct meanings inside education.

Compounding the collective ataxia is that terms are often employed inaccurately by less-than-qualified individuals. “Automaticity” has been reduced to “reaction time,” though it used to be a much more robust concept. “Self-efficacy” is frequently confused with “efficacy,” despite Pajares’ and Bandura’s effort to maintain a modicum of purity in their theories. I believe these two forces – the preponderance of pseudo-neologisms (old words with new meanings) and collective frowsy phraseology – hinder our field.

With this understanding I coin my own addition to our field’s jargon. I feel no hypocrisy in so doing because, contrary to my points above, this new word a) does not change the meaning of an exiting term, and b) serves to constrain its meaning to its theoretical origins.

Five decades of philosophy on the concept of validity-in-assessment has yielded several meaningful changes to how the term “validity” should be used. Specifically, it is not an assessment that is valid, nor can validity be assigned to the assessment’s results; it is the specific manner in which we place meaning on the assessment results (our inferences or interpretations) that can be more or less valid.

There are two massive practical problems with this view: First, it takes a lot of breath to refer to the “validity of the interpretations of assessment results,” so many au courant experts still call specific tests valid or invalid. It’s just their shorthand reference to validityoftheinterpretationsofassessmentresults. Second, because the great validity theorists saw themselves as refining the existent term, they had no need to add modifiers or create other variations of the word “validity.” (e.g. We don’t hear Messickian validity contrasted with Cronbachian validity, etc.)

VIAR, a pronounceable replacement for “validity of the interpretations of assessment results,” overcomes these two deficiencies. It is easier to type and say (“vi-yer”) than the phrase it represents, and its acronymous state encapsulates the parsimony and coherence of the modern view of validity. Additionally, it does not aggravate either condition I describe above: It is a new term (not a redefinition of an old word) and remains true to its founding tenets.

I don’t know if the assessment community will embrace VIAR in their speech or writing, but doing so (or finding another suitable term) would dispel much confusion.

I became re-interested in Multiple Intelligences (MI) theory a couple of days ago when a friend posted this link to his Facebook:

http://www.youtube.com/watch?v=tDtZEpf_SJ4

It’s Dr. Howard Gardner explaining the last 25 years of work on MI – a term and theory he invented. I admit I’ve only watched the first third, but I was summarily unimpressed with the general tautology of his descriptions. Further, after I heard his “Scientific Claims” of MI, I had to question whether he even understood the meaning of “scientific.” I wasn’t sure if I was correct, crazy, or just just too proud to accept the theory on which much of my colleagues’ work is based. So I turned to the literature.

A quick search found several empirical and theoretical papers that echoed my thoughts. Curiously, it seemed the Canadians held the most disdain for MI. Two of the articles (Klein, 1998 and Visser et al., 2006) included rebuttals from Gardner himself, followed by responses from the original authors. In both cases Gardner accused the researchers of not truly understanding MI. Visser responded beautifully (while backing up my observation of MI’s non-science status):

Gardner states that we have failed to grasp the core of MI theory, and perhaps in some sense he is right: it remains unclear to us what it is that MI theory can explain about intelligence, above and beyond what has already long been known. Gardner could clarify this “core” for us, by providing falsifiable, testable, MI-based hypotheses that would predict results different from those predicted by existing models of the structure of mental abilities. We encourage Gardner to provide “intelligence-fair” measures for his eight “intelligences” – tasks involving no extraneous personality, emotional,or sensory acuity content – so that MI theory can again be put to the test.

Because I’m struggling to get my head around it all.

1. The Wiki for Evaluation and Assessment: I got fed up with textbooks, so my students are writing their own this semester. Teams of students have been assigned two chapters to write and provided access to my personal library of assessment texts. They’re expected to post a finished draft for each chapter the week before we cover it in class. I’m writing the first three chapters. The first chapter on reliability they read (and critiqued to shreds) this week; it needs severe revision. The second and third chapters (correlation and validity) are in the works and will be done this weekend.

2. Completing a review of the video analysis literature. We’re trying to use standardized video experiences to replace a small portion of field experience. We hope to overcome the paradox of using non-standardized field experiences to fulfill standards. We’ve been blocked by those who fear the state would never allow it, but I just learned that the new state commissioner of education, David Stiener, has a background in video analysis of teachers. Also, I’ll be meeting with Michael Preston of Teachers College at TEDxNYED in March. I need to get something together because my abstract was accepted to EDMEDIA Toronto in June.

3. I’m on the College Technology Council, which has sent representatives (including myself) to form a working group to decide what to do about our CMS going away. (It was bought out by Satan, I mean, Blackboard.) I’m on a committee for that working group that is assessing how faculty have used the CMS in the past (to form criteria for the next CMS), and a sub-committee that is looking into emerging technologies. Yes, it goes Council→Working Group→Committee→Sub-Committee.

4. I am now the chair of the unit assessment committee. That’s the group that design the data collection for all of the college’s teacher education accreditation needs. We’re currently developing a more robust system to assess the graduate-level teacher education outcomes.

5. I’ve completed an analysis of foreign language enrollments based on government databases. I know it was wrong, but I did it without completing a lit review. Now I have to go back and finish that before I can submit the paper.

6. I’m conducting a lit review of language students’ motivation for another paper based on six years of national surveys submitted by Middle East language learners.

7. SUNY Brockport and CUNY Cortland won a Title Vi grant to increase globalization instruction at the GE level. They’re including a dual-site Chinese course, so I promised to hook them up with the efforts in Utah (BYU-Utah Hindi, and BYU-UVU Chinese). I also owe the PI an email explaining my concerns with their rather arbitrary comparison level on two groups of students…

8. The SSRC wants to continue our relationship, which is great, but I’m very busy. I’ll respond to their email this weekend when I get a moment to breathe.

9. A dear friend has asked me to check out his work on Brain Honey. That should be interesting.

10. Oh, and somewhere in there I’m teaching three preps this semester. We’re three weeks into the semester and one course doesn’t even have a completed syllabus yet!

11. Post next week’s EDI 419 activities for enterprising students who want to take advantage of their week off.

12. I almost forgot: I need to write an evaluation protocol for a massive grant application the NMELRC is turning in.

The New York Times reports that President Obama will call for an overhaul of No Child Left Behind to fashion it after his own “Race to the Top” initiative. (Why not then just revoke NCLB and pass laws called “Race to the Top?) Unfortunately, I fear RTT (RtT? R2T?) will just have different shortcoming, loopholes, etc.

I already see one glaring issue with RTT, which prompted me to submit this letter:

President Obama wishes to codify his “Race to the Top” program by overhauling the No Child Left Behind laws (NYT, January 31, 2010). This is a welcome attempt at fixing a broken system, but as a teacher educator trained in the issues of assessment, I have severe reservations about one aspect of the program.

Race to the Top specifies that student test scores must be used as criteria in judging teacher competence. The problem is that the state tests were not developed to measure teacher ability. While theory would suggest a link between student performance and teacher quality, reality is much more complex and nuanced.

The 1999 Standards for Educational and Psychological Testing, published by the American Educational Research Association, the American Psychological Association, and the National Council for Measurement in Education, require evidence to support each intended use of a test. Before we require states and districts to use students’ test scores to judge their teachers, we must first establish the degree to which student test scores are dependent on teacher quality and not on geographic location, socio-economic status, parental involvement, race, ethnicity, gender, and other traits that are beyond the teachers’ control.

Some readers may wonder if there is even evidence to support judging a student by their performance on the test and, in reality, many state tests do not fulfill the requirements set forth in the AERA/APA/NCME Standards. But the issues would be compounded were we to extend already unsupported interpretations of test results to include individuals who did even take the test.

I doubt it gets published, but I think I will start speaking a little louder about the evils of what James Popham has called “Second-step inferences.”