Archive for January, 2011

[2b2k] Guardian aggregates all its data

The Guardian has been publishing its data for the past couple of years. Now it is making all of it available in one spreadsheet:

Want to see all of the data we have reported? Here’s all the data we’ve covered over the last two years, that’s almost 600 spreadsheets linked from one spreadsheet

Not just transparnecy, but convenience! Well done, Guardian!


If we had called it the Age of Patterns instead of the Age of Information

Claude Shannon, a father of Information Science, had to call the differences that move through telephone wires something. He picked “information,” a term that had meant, roughly, something that you hadnt known, or the content of written tables. Had he called it “data,” or “patterns,” or “differences,” or “Arthur,” we would have skipped right past one of the false continuities: from information to knowledge. We would have had the Age of Patterns, characterized by an abundance of patterns of difference, and we wouldn’t have thought that that has anything much at all to do with knowledge. But, because traditional information had something to do with expanding what we know, we tricked ourselves into thinking that our modern technology is about making us smarter. With an abundance of information, it seems we must be gaining more knowledge. With an abundance of patterns, or differences, or of arthurs, it would not have seemed so.

The new age is one of connection. This is less misleading, for it has us looking for its effect on how we connect with one another, how we connect our ideas, and how we connect our connections. And these are, I believe, the right places to be looking.


Visualizing Wikipedia deletions

Notabilia has visualized the hundred longest discussion threads at Wikipedia that resulted in the deletion of an article and the hundred that did not. The visualized threads take on shapes depending on whether the discussion was controversial, swinging, or unanimous. For those whose brains can process visualized information (as mine cannot), you will undoubtedly learn much. For the rest of us: Oooooh, pretty!

They’ve posted some other analyses as well. For example, “The analysis [pdf] of a large sample of AfD discussions (200K discussions that took place between November 2002 and July 2010) suggests that the largest part of these discussions ends after only a few recommendations are expressed.” And: “Delete decisions tend to be fairly unanimous. In contrast, we found many Keep decisions resulting from a discussion that leaned towards deletion…”


[2b2k] Amateur astronomers, and science as a network

I met today with Aaron Price, who’s with the American Association of Variable Star Observers, a group celebrating a hundred years of gathering data from amateurs and professionals about variable stars. AAVSO has an archive of over 19 million variable star observations. Aaron is particularly interested in enabling and encouraging amateurs to become increasingly involved in the scientific process, ultimately collaboratively writing publishable articles. (I’m putting this my way, not his, so don’t blame him for my infelicities.)

We talked a bit about who should be called a scientist. My own view is that if you have this discussion without any context, then you look to paradigmatic scientists — works in a lab (perhaps), designs and runs experiments, formulates hypotheses, has academic credentials, wears a lab coat. In such cases, when there is no actual need driving the question, arguments about edge cases can’t be resolved. On the other hand, if something hangs on the question — does the person get funding, get invited to address a conference, is allowed access to equipment, get to claim a particular standing in an argument, etc. — then the question is more likely to be settle-able. For that reason, most discussions about whether citizen scientists are scientists (or, are “citizen journalists” journalists, etc.) should be addressed (in my opinion) first by asking, “Why do you ask?”

This seems to me to be an illustration of the way everything (well, almost) is becoming a network. In the old days, when science was a lot like publishing, the line between scientist and layperson was fairly well (but certainly imperfectly) drawn. In a networked world, it’s not simply a matter of redrawing lines, so that now citizen scientists are inside the Circle of Science. Rather, the nature of the lines is different. All members of a network are connected. The question is the nature of the connection, and that can change instantly based on interests, skills, credentials, and the project underway. The old lines disconnected; the new ones connect. And that makes it far more difficult to come up with persistent answers to questions like “Who is a scientist?” or “Who is a journalist?”

Often, in a networked world, it’s better not to insist on an answer. More important than deciding exactly who is inside the charmed circle is figuring out how to make the network smarter — which almost always means extending the network as far as it can possibly go.