As we’re packing for our move, we’re having to decide what to keep and what to throw out. I found my old cahier from when I taught in France, and I was about to throw it away (I’m not going to be teaching French, especially not with that text, anytime soon), but I still flipped through it.

I found three pages at the back of the notebook that were research notes for my paleography study on Ste. Marguerite’s confrontation with the dragon. From these notes, its easy to see why I sometimes wish I had done my graduate work in the Classics, and gone on to become a medievalist. My teacher (who’s back in France now) once spent a summer traveling around Europe to transcribe the 19 existing copies of an epic love poem.

Just from this one afternoon with a card catalog, I could have justified a similar adventure. There’s even a reference to a manuscript at the Pierpont Morgan Library in New York.

Page 1
Page 2
Page 3

I’m currently writing a mass mailing PHP script and it makes me feel dirty.

The NMELRC has been running an online survey since before we went on study abroad, and now they want to send a short follow up questionnaire to those who submitted their email address and agreed to be contacted at a later date. So, technically, it’s not spam, and we have no intention of sending out repeated emails. The reason we need to automate the process is that we want to personalize the emails based on the responses on the previous survey and there are over 1,000 addresses.

It still makes me feel icky though like selling out and going to law school or something.

I owe a lot of updates to you all. Here’s the quick list:

My Washington-New York City-Baltimore trip went well, but was very tiring and I’m afraid it brought some serious illness with it. After being laid up for a weekend, I started to catch up on everything in time to fly to Upstate New York for an interview at SUNY Brockport. That went well, and I will definitely consider it if they offer.

I flew home on Saturday, it’s now Monday, and I fly out to present at SITE tomorrow. Yeah, busy.

The report of the National Research Council evaluation of Title VI and Fulbright-Hays (part of the Higher Education Act) will be released tomorrow. I participated as a “data analysis consultant,” and I’m cited in Chapter 11 (or I was in the last version I proofed). For those interested, the public hearing will be webcast (details).

The enormity of working on that project for me cannot be overstated. It was fortuitous that they contacted me, and it led to my invitation to speak at the U.S. Department of Education IEPS workshop last month (part of the abovementioned tour). I met some truly influential people, ate some wonderful food, and gained more in knowledge and experience than I did in money (though they paid me well).

This summer will be a busy time for us. I knew I was heading off to interview at the University of North Texas on May 8, and I knew that the SSRC was trying to contact me. I thought the SSRC wanted to talk about the appendix I wrote for their National Research Center report, which was on the limitations of the EELIAS database. What I didn’t know was that the SSRC wanted talk about continuing my consulting work with them, and referring me to another research group.

So, in the matter of two days, I added trips to D.C. May 17-19, New York June 2-3, and Woods Hole, MA, sometime in July. I’ve already contacted a old Arclite friend, Brant, in D.C., and extended my stay in New York by a few days (June 1-7), to spend some time with my parents. Brooke and the kids will be flying out around the same time, so we’ll hang out together. We thought about going to a Yankees game in New York, but they’re playing the Red Sox, so the cheapest tickets left are $40.

On campus, I’ll be finishing up my last project and defending my prospectus, while working for the NMELRC. My work there will include evaluating the current state of Arabic Without Walls and porting Dr. Dil Parkinson’s ArabiCorpus to Hebrew.

About four months ago, Brooke was browsing the job ads for my field, and discovered that UNESCO was hiring a new director for their International Education Bureau. The only real requirement was fluent English proficiency, and French was a bonus, so we filled out the online form. In all honesty, it was late, we were a bit slap-happy, and, although we responded truthfully, we got a kick out of the application. (Just the dream of returning to Paris was entertaining enough.) We didn’t expect to hear back from them.

Of course, we didn’t.

Today, however, I received an email informing me that the post had been reopened, and I “can re-apply for the above-mentioned post if [I am] still interested.”

I figure good laughs are good twice, so I’ll give it another shot.

Well, my work with EELIAS continues. Our contact at DOE (are acronyms supposed to have definite articles?) asked for some feedback, so I wrote a 15-page informal report on the issues surrounding the system. I’ve identified several technical issues, which I classify as minor because while they create hoops the users must jump through, they do not limit the usability of the data.

However, I have also identified major issues, ones that severely inhibit EELIAS data being used, which I place under the headings Accessibility, Accuracy, and Meaningfulness.

I’m also thinking about turning in this report as part of my evaluation course.

http://enworb.byu.edu/documents/ed_report.pdf

I figured it was time to put my CV up for public consumption. Here it is. Of course, it’s a work in progress, and I will probably alter it depending on the position for which I am applying at the moment, but this is just a place to point people who might be interested.

Unfortunately, I heard back from Utah State and, due to budget cuts, they’ve had to pull their opening back to next year. It would have been a great place for me to fit in, though I don’t know how we would’ve handled living in Logan.

I’m using this in work, and I’m having to explain it a lot, so I’m writing this entry to help myself clarify and simplify my personal understanding of this principle, as well as to have a place to send people who ask.

Pearson’s Correlation Coefficient (r, for samples) is the degree and direction of relation between two variables and ranges from perfect positive correlation (1.0) through no correlation (0.0) to perfect negative correlation (-1.0). If the correlation is positive, the first variable increases or decrease as the second increases or decreases. If the correlation negative, the first variable increases as second variable decreases and vice-versa.

Correlation can be understood, to an extent, by rank order of the first and second variables. If Johnny and Jane score 75% and 80% respectively on a pretest, and then 85% and 95% respectively on the post test, r would be a perfect 1.0 because Johnny’s score was lower in both cases. This would hold true even if Johnny scores 94% (still lower than Jane’s) on the post test, but not if his post test score exceeds Jane’s. At that point, r is a perfect -1.0. With only two cases, r will always be 1.0 or -1.0, but of course, such clear cases are rare.

Extreme r can be visualize through plotting a line for each case from their first variable value to their second variable value. Here are three graphs for simplistic 1.0, 0.0, and -1.0 correlations. It’s a bit more difficult to spot non-extreme correlations with large numbers of cases:
166 cases, r= .71
166 cases, r= -.70
166 cases, r= -.13

For a real world, example, we looked at two variables (intensity of study and score on admissions test) for 166 Middle East language students. We found there to be basically no correlation between the two variables (r=-.04) Here’s the graph, and a close up of the chaos.

Notes:

It’s cliché, but correlation does not mean causation. The sun coming up may correlate with an increase in sewer pressure, but the sun has little direct effect on this. All r indicates is how to variables relate to each other, not what causes this.

I once heard an M.D. on the news say, “80% of type-2 diabetics are obese, so it doesn’t take a Ph.D. to see the connection.” Well, actually, I guess it does. A Ph.D. (sh|w)ould point out that the two conditions may actually be caused by a third, present in both cases. (I’m not saying obesity doesn’t lead to diabetes, just that this statement was not a good argument for it.) Sadly, we, as a society are conditioned to think in terms of percentages, “grade levels”, etc. And very few of us are not convinced by their face validity.

Correlation cannot be determined if the sample size is 1, because the standard deviation of a single sample is 0 and to compute correlation, one must divide by the standard deviation, and one cannot divide by 0.

r can be tested for significance, but its square is often considered more important because it can be interpreted as the percentage of variance in the second variable that can be accounted for by the first (and vice-versa). Thus, r², is called the coefficient of determination.

Last Summer, the Social Studies Research Council, who is charged with evaluating Title VI-funded programs, decided they could do their job better if they had access to the EELIAS database. EELIAS, started during the 2000-2001, is a database in which all programs funded under Title VI are to log their enrollments, expenses, etc. It’s a good idea, but poorly executed.

First, they contracted out the creation and maintenance of the system to a private company. Now, whenever the DOE wants some data, they pay that company to produce views and aggregations. This company has it pretty sweet. There doesn’t ever seem to be any assurance of data integrity, and there is no oversight on the accuracy of the data as provided by the schools.

So, the SSRC wanted the data for what it was supposed to contain, but didn’t know what to do with it. As we are also interested in enrollment trends, we said we’d take a look at it for them. So, they handled the FOIA request for the data, and sent me a 115MB Oracle dump file.

After weeks of work, I got what I could out of it. Some schools listed sections of Arabic 101 under the language “Other”, while some reported multiple sections with zero enrollment. My first look at incoming grants listed the average grant amount as $1.70. (Some data entry people had mixed up the fields for number of grants and total amount.) Finally, I thought I had something to show, though it was of no use to us.

I went to New York to present what I had found, and the difficulties we had finding it, to the SSRC. I had a slide that said, “This data was never meant to be looked at.” It seemed that some committee had mandated that Title VI funds be accounted for, so they set up a database to record, but didn’t build it to ever be used. Even the data I had was not acceptable to the SSRC, and I couldn’t blame them. Most of the issues were called out by an astute sociologist from the University of Chicago, who kept asking what the definitions of the terms were. Terms like “lecturer” mean different things from school to school, as does “enrollment”, and there was no clarification provided on the EELIAS form.

The SSRC was very gracious and thankful for my time. They asked me to write an appendix for their report detailing the shortcomings of this database. Such a chapter would be an ace up their sleeve should any opponent use EELIAS data against them.

Upon returning we sent out Fall enrollments to some school where we know people, just to see how accurate it was. Only one out of the five schools found any errors. Then we realized that we had sent out the data with the wrong years. (The reports are filed after each academic year, so Fall 2000 would be filed during the Summer of 2001. The dates on the data were the dates the reports were filed, so Fall 2000 would have been indicated as Fall 2001.) Sure enough, that the numbers from the one school that did find inaccuracies lined up much better once we fix the years, but the fact that four out of five schools reported that the data was accurate when, in fact, it was not, demonstrated to me the source of much of EELIAS’ error: The schools didn’t care about entering accurate data the first time, and they didn’t care when we asked them to verify the accuracy.

I wonder if it’s unethical to submit data for verification, but include some known inaccuracies as a check against the verification.

With the rolling over of the site, I guess I need to give an update of what I’m doing for employment.

During the Summer, I work full-part-time (40 hrs/wk) as a measurement specialist for the National Middle East Language Resource Center (NMELRC) where I fight a constant battle to nurse data out of complex data sets and draw conclusions that are both statistically and practically significant. For example, I currently have access to a HUGE database from the department of education. We had been hoping it would answer several lingering questions, but after we ran the preliminary data we had extracted by someone familiar with the real numbers, we were told they could not assure the accuracy of the data.

One great part of this job was that, after an argument with the department secretary, we decided to put me on contract so as to bypass her (all time cards had to go through her).

During the school year, I cut my hours at the NMELRC to 10 and work the rest of the time teaching Technology for Educators. Basically, this is a teacher ed course where we show teaching majors how to properly integrate technology into their classroom teaching. There is plenty of importance there, and my dissertation will focus on measuring changes in the students attitudes.

Further I do some freelance work developing measures and instruments for other grants. That is the primary source for my bike fund.