Tag Archives: James Gleick

The Information: But What Does It Mean?

Cover of "The Information"

Title: The Information: A History, a Theory, a Flood

Author: James Gleick

Publication Date: 2011

LC Call Number: Z665 .G547 2011

Shannon’s information theory is very much the sort of theory you’d expect an engineer to produce.  He carefully removes meaning from the equation and defines information as the ability to recreate the same signal at one end of the line that you put in at the other end.  But he also noted that if the signal is a message, it generally has a high degree of redundancy in order to guard against “noise,” which interferes with messages, and possible loss of information in the course of transmission.   He didn’t consider this redundancy to add any information. If you see a “q” in English, you can be fairly certain that the next letter will be “u” and you won’t gain any new information when you see it.

This is curious, because it means that a random arrangement of letters (or numbers or other symbols), which is the least predictable, carries the most information.  But at the same time, random signals are very likely noise, and are exactly what you don’t want to transmit.  In fact, in some ways the redundancy itself is what tells us that something is a message.  We look for patterns and if we don’t find them, we know that we can ignore things.

So getting from information to intelligibility was a problem that Shannon avoided entirely, but if you put it back in, this all gets very complicated and messy.  Margaret Mead asked him about how body language fits into his theory; it’s an interesting question, but Shannon brushed it off by saying that he was not interested in meaning. But asking this question opens up all sorts of other questions.  The meaning of a message does not depend only on the symbols that are used.  Most of it comes from the context in which they are presented. Language is a fairly trivial example. If we are doing the exercise I mentioned above, in which a reader attempts to predict the next character, I can be fairly certain that I would make much better predictions for English words than for, say, German ones.  And this is still staying within the same set of characters!  For a Hebrew or Chinese text, even if I were given a menu of characters from which to choose, I’d do no better than random, as I’m not familiar with any of the symbols used  nor the rules by which words and sentences are formed in those languages.  So, even if we attempt to take a very scientific view of information and measure it quantitatively… we really can’t, because the amount of information varies based upon who is measuring it!

So intelligibility is a really complex thing, and very difficult to nail down. At some times, it seems as if the book walks right up to the very threshold of the moment when the information is about to be understood, and just leaves it there.  You’ve got Turing coming up with the Turing test, which turns the question of artificial sentience into a behaviorist one; you’ve got Shannon scolding people who try to think about what “information” means outside of computer science and trying to apply the concept to their  own disciplines.  It gets a little frustrating realizing that there’s no way of identifying what makes something meaningful or intelligible—that’s one mystery that nobody in the book really even attempts to address.  There is a long discussion of random numbers, though, and identifying patterns in them, which turns out to be really the same question, or at least a closely related one. It’s difficult to say whether a number is random or whether it means something. What, they ask, is the smallest uninteresting number? If it’s the smallest uninteresting number, is it really an uninteresting number? What about Borges’s Library of Babel, with every meaningful and meaningless book included in it? Am I making a deliberate reference to something totally unrelated with the title of this post? (It depends partly on whether you think I’d do that, and partly on whether you find any connection between those words and some other words. Coincidental? Hard to say!) You can often feel fairly certain that a book constitutes a meaningful message (although you can never prove it isn’t just a random combination of signs), but you can’t prove that one isn’t.

Where am I going with this? I’m not sure at all.  At the very least, it seems to prove that the meaning does not inhere in the information, or maybe anywhere in particular. Although it seems that meaning is something that exists, apart from the arbitrary systems which humans use to represent it, it’s strongly hinted in the book that there’s no consistent rule that will tell us whether something is meaningful or not.

I’m sort of fumbling with this and not expressing myself well, but the more you think about the question of what meaning is, the weirder it becomes.

2 Comments

Filed under Professional or Philosophical

The Information: Always Already Literate

Cover of "The Information"

Title: The Information: A History, a Theory, a Flood

Author: James Gleick

Publication Date: 2011

LC Call Number: Z665 .G547 2011

I love the history of technology.  I’ve barely dipped my toe into it so far, but I really enjoy seeing how certain technologies developed and particularly what they meant at the time.  I’m interested in this in lots of different contexts, but I especially like the history of information. I knew a little about it going in; I expected Claude Shannon, who invented information theory, and probably should have expected Alan Turing as well. I certainly wasn’t surprised by the appearance of Ada Lovelace and Charles Babbage, either. The stuff about the telegraph was actually briefer than I expected, and the explanation of what Morse actually contributed to it was of great interest—I was already somewhat familiar with the early optical telegraphs and so on.  However, I was taken aback by some of the mathematicians and scientists: Kurt Gödel? Johannes Kepler? Richard Dawkins? …but perhaps I shouldn’t have been. And then there were some people with whom I wasn’t familiar at all, like Norbert Weiner and Gregory Chaitin. I enumerate this list not to show off, but to give some idea of what I knew when I started the book, and where it fits in for me.

I think that I’m really close to being The Information’s target audience.  It’s intended to popularize this history, and I think it could be read by anyone who is interested enough in the subject matter to pick up a largish book on the topic in the first place.  However, having a little prior exposure to some of the pieces of the puzzle adds an extra dimension to a book like this, that is to say, a book that helps to put it all in context. It’s very wide-ranging and not strictly chronological. Instead, it follows the threads of some important notions like the meaning of information, the importance of randomness, and so on, so that these same figures keep popping up again and again.  Shannon is really at the heart of the book, which pleased me, because I had had only minimal exposure to his theory in library school and really wanted to know more about him.

Far be it from me to try to cover the whole of this book. There’s so much ground covered here that it feels longer than it is, and there’s little use in my trying to outline the important ideas. Instead, I’ll do what I always do, which is to pick up some of the threads that are most interesting to me.

Near the beginning of the book, as Gleick is explaining how we came to view information as a thing in the first place, he takes a detour into the rise of literacy.  This was intriguing to me, not only because I immediately developed a little crush on Robert Cawdrey, whose Table Alphabeticall, conteyning and teaching the true writing, and understanding of hard usuall English words was an early proto-dictionary and had the explicitly educational aim of helping inexperienced readers make sense of their books, but also because it is so astonishing to consider the obvious fact that, before dictionaries were invented, there weren’t any.  It’s very difficult to imagine not thinking of words as components of a dictionary, but this is only a step toward trying, as a person from a literate culture, to imagine what it is not to have writing available as a tool.  Of course, there are still nonliterate cultures in the world, but in most of the world, even people who don’t have strong literacy skills are still heavily influenced by writing culture all around them.

In a way, we (for some value of “we”) are always already literate.  I have no idea how I perceived the world before I learned to read, not only because I didn’t yet have the ability to form long term memories (though I don’t suppose that helps!) but because literacy is such a major and unavoidable component of the way that I was taught to think that it’s impossible to get away from it.  I’m reminded of the character in If on a Winter’s Night a Traveler who had taught himself not to read. Gleick writes:

When it comes to understanding the preliterate past, we modern folk are hopelessly automobilized.* The written word is the mechanism by which we know what we know. It organizes our thought. We may wish to understand the rise of literacy both historically and logically, but history and logic are themselves the products of literate thought.

*This is a reference to Walter Ong’s comparison of our attempt to understand oral literature in terms of written  literature to thinking of horses as automobiles without wheels.

The relationship of history and logic to literacy is discussed further in this part of the book. As there’s some discussion of the role of the oral tradition here, I don’t believe Gleick is being dismissive of the role of the oral tradition in nonliterate cultures. However, the way that literate people understand “history” is as something intimately connected with writing; that is the technology that makes a stable and continuous historical record possible.  Formal logic, too, is shown to have arisen with writing.

So this is almost exactly the problem that Butler found so interesting—we can’t get a good view of what literacy does to the way we think, because it’s how we are thinking.  It makes me wonder how culture might have developed if some other means of preserving and transmitting it had been transmitted, or if none had. Sounds like a science fiction novel in the making, if an author could get his or her head around it.

2 Comments

Filed under Professional or Philosophical