I usually have very high expectations for conferences. It’s easier to hold myself to high expectations when I hold everyone else to the same standard. That said, there are different levels of conferences. Some (like AERA) are hard-core research that require data to be gathered before they’ll consider you as a presenter. Others (like PIDT) are nothing more than brainstorming sessions. SITE falls uncomfortably somewhere in between those two extremes.
The Good:
I made some really good contacts with people who are research topics that were very close to my own. I came to a tentative cross-administration agreement with a grad student at Wyoming (I’ll give his instrument to my students and he’ll give mine to his), was asked, “How did you not get an interview?” by a faculty member at a big school to which I had applied, and developed an interest (though minor) in the cultural imperialism of educators assuming their definitions of citizenship and ethics applies to the online world. I also had two people from good schools tell me to hold off accepting offers because they will have positions open in a few months.
The Bad:
The keynotes were way more flash and fluff than substance: pleasant experiences that left me wonder what I had learned. The speakers either showed a bunch of wizzbang tech that “someone” was using without evidence of success, or brought up issues in the field without offering solutions. Specifically, one educator showed enrollment numbers and proclaimed they were moving “learning” online. No, he showed they were moving “people” and “credits” online. Without assessment (formal or informal) he can’t say they were moving learning. Another stated that we won’t change any policymaker’s opinion with research. Um… Yeah…. What do you suggest?
The conference was not organized thematically. Even though the topic was listed next to each presentation, the sessions usually consisted of presentations from two or three different topics and often two presentations from the same topic would present simultaneously in different rooms. It meant a lot of people were coming and going throughout each session.
The Ugly:
I attended a presentation of research that compared preservice teachers’ “confidence” to their competence in certain information literacy tasks. Their method was to survey a large number of student teachers to establish their level of confidence in completing certain information literacy tasks, and then to observe whether or not they completed those tasks. The conclusion was that preservice teachers are overconfident in many important tasks.
I posed the same question to this researcher that I had to graduate students who presented on “confidence”: “Given a psychometrically complex trait like ‘confidence,’ I’m wondering how you operationalized it, and how you measured it. Specifically, you results table showed two possible responses: ‘agree’ and ‘disagree’. Were those the only two response categories, or did you flatten the responses into those?”
His response was quick and to the point. I won’t quote directly, but it was something like: “We simply used the term ‘confidence’ in the items and they were ‘agree’ or ‘disagree’ responses. We know it would probably not hold up psychometrically. Next question, please.”
Whoa, whoa, whoa. This researcher conducted a scored survey of over 600 students, but chose not to first research whether someone had developed a framework to measure the trait he was investigating. He admitted his measure was probably flawed, but then drew his conclusions from comparing it to another poorly developed measure (of competence).
Why did this get me upset when other presenters had done the same thing? Because this was one of two award-winning papers at the conference. I don’t mind that some presentations at a “mid-major” academic conference would include quasimodo research or brainstorming sessions, but for the conference to bestow an award on a paper that did not even define its key variables is a good indicator of its quality.