Dave's book review for The Art of Doing Science and Engineering

This was excellent! I already knew that Bret (worrydream.com) was a good at explaining things. This lucid introduction to "Hamming on Hamming" is no exception.

(Tangetial note: when I look for PDF copies of influential computer science papers, I’m amazed at how often my search results in a link to Bret Victor’s website.)

Though I later found that Victor was re-stating much of what Hamming himself says about this book in the Preface, Introduction, and Chapter 1, I think Victor’s introduction is more fun to read - and it also has the benefit of being able to talk about Hamming in a way that Hamming himself cannot because he’s the author in question.

The first chapter is really another introduction to the book as a course of study. The goal is meta-education (book subtitle: "Learning to Learn").

As warned, there is a little math. It’s an example "back of the envelope" calculation of the explosive growth of new information to be studied in each new generation. This math digression relates to the chapter, but the point is not really to drive home how we arrive at some numbers, but rather to illustrate that this calculation is the sort of thing that great thinkers might do.

And given this explosive growth of new information, Hamming instructs us to concentrate on "the fundamentals". And how do we figure out what will be "fundamental" in the future? By trying to predict it!

Of course, no one can predict the future. But by having a personal "vision" of the future, you can steer yourself towards a goal even as new information comes to light. There’s a tension between learning essential new things, but also not chasing every shiny object that crosses your path.

Lastly, having a vision or goal is a way to steer towards "total happiness rather than the moment-to-moment pleasures."

I like this idea and I can see how it would be profound advice to follow!

Right in the first sentence, I’m confronted with my ignorance of mathematics and physics. Hamming says,

I had to look up the word soliton.

Ha ha, I’m being unfair to the Wikipedia entry. In the first paragraph, it says essentially the same thing as the Wiktionary definition.

Further, it had this nice quoted passage from John Scott Russell writing about his discovery in 1834:

Solitons appear to come up in a variety of physics settings (fiber optics, magnetics, nuclearreactions, and even a sort of science fiction spaceship warp propulsion known as the "Alcubierre drive" which is a, to some degree, a product of mutual inspiration between Star Trek and physicist Miguel Alcubierre - I’ll have to keep an eye out for that one now that I know about it). But as near as I can tell, we haven’t proven or disproven Hamming’s prediction on this one yet.

Moving on to the second sentence (and beyond)…​

The chapter is about the move from continuous (analog) signaling to discrete (digital) and why this has happened.

The first reason is that analog signals need to be amplified in order to be transmitted over a long distance. I remember reading an anecdote about early attempts to send voice signals across the Atlantic ocean by cable in which, if I remember correctly, the amplification produced a hopelessly messy blast of sound and also melted the cable!

(TODO: For the above, am I thinking of the Neal Stephenson article in Wired "motherboard, mother earth," I think? Or was it the Claude Shannon biography???)

By contrast, digital signals are vastly easier to transmit long distances because we can clean them up and even correct them for errors as we pass them along, producing an exact copy on the other end.

Also, here’s a fascinating way to phrase a fact about digital information:

Oh, and this tantalizing bit:

(I only know a little bit about analog computers that calculate using continuous electrical logic rather than binary/digital logic. And what little I know makes me wonder if maybe they’ll make a resurgence in applications where they provide the best low-power solution for a computation? Obviously, we would pair the analog circuits with digital circuits to make a hybrid computer in whichever ways make sense. Also, we can compute with mechanics, fluids, molds, fungus, and folded paper. So who knows what the future will bring?)

Second: Digital is cheaper. Especially once we figured out integrated circuits.

TODO: link to Crystal Fire, once I get that book review up

Third: Information is increasingly what we work with rather than physical goods.

Also: Robotic manufacturing and engineering (with increasingly complex products that require increasingly complex maintenance), increased data-crunching power for simulations used in science, especially really costly stuff like atomic bombs.

(Hamming makes a good point about over-reliance on simulation in science - he was an early advocate, but he’s well aware that for hundreds of years, "Western" thought centered around too much book learning and not enough direct observation experimentation in the actual world. You always have to go test things in the real world to see if the simulations are correct.)

Societal effects, people management, and central planning:

Hamming goes at length on the evils of micromanagement. That was a big thing in the 1990s! I actually kinda remember that. I started my work life at the very tail end of the 1990s. It’s really interesting to contrast that problem with the state of affairs today. I’m not a business person by any stretch, but I don’t think micromanagement, per se, is the biggest problem we’re facing with huge corporations anymore. I have thoughts. But to say anything further would be far more political than I’d like to get in covering this book!

Another Hamming prediction that caught my eye, but my fellow book club pal really hit right on the head (github.com) with:

Yeah, my thoughts exactly!

Entertainment: Well, computers now completely dominate here and Hamming has been proven 100% correct. I’m fascinated (and, if I’m honest, somewhat dreading) the "AI" stuff he hints will be coming in Chapters 6-8. But it will be interesting since he wrote this in 1996!

Warfare: Hamming writes:

Though we’ve had remotely-piloted and autonomous military aircraft for quite a while now, I’m writing this in 2025, at which point we are at the beginning of a whole new chapter of warfare: with inexpensive quadcopter and water craft "drones" boats accounting for a huge percentage, perhaps even the majority (?), of strikes in the defense against the Russian invasion of Ukraine. The potential is terrifying. But maybe this stuff will save lives in the long run? I have no idea.

Back-of-the-envelope calculation break! Hamming does some calculus here, which I have to assume is really great and totally satisfying if you’re into continuous mathematics. I’ll be perfectly honest. I’ve never found modeling rates of growth to be all that interesting. Especially something as abstract as "innovation". Actually, the more I think about it, I find this particular exercise actively boring, in a way that has nothing to do with not being able to easily follow the mathematics.

Lastly, Hamming makes a recommendation/prediction: General-purpose ICs will tend to be a better choice than specialized custom ICs. This is based on the rate of innovation, the advantages of shared knowledge, and the price advantages of scale. Good job, Ham Man! Absolutely correct. In 2025, there are a vast number of different microcontrollers and other ICs available off-the-shelf and it is extremely rare to need something custom.

I’m excited about the next chapter, which will be about the history of computers!

Here we go!

I know of the abacus (I actually had a little one as a kid but at that age, I couldn’t make heads or tails of the little instruction book that came with it).

But "sand pan"…​ I can’t find any reference to that type of instrument, even on Wikipedia. Even Marginalia Search (marginalia-search.com), which does real "exact match" web searches, didn’t turn up anything useful. (Just a military operation of that name, gold panning equipment, geological surveys, a child’s craft activity for Biblical studies, and barbecue techniques.)

And then he says that the next big leap in calculation came from John Napier’s invention/discovery of logarithms (in the form of tables), and from there, the slide rule.

I find the subject of logarithms to be quite fascinating and I’ve long felt that I should make a study of them specifically because of how often they come up in computer science and the history of computation (the "big-O" notation, slide rules, etc.). So I’ve made a stub card to expand at some later date: Logarithms.

Next come the mechanical computers - typically very specialized for a particular task. I find these fascinating as well. Mechanical comptuers always make me think of The Clock of the Long Now (wikipedia.org), which contains a mechanical computer to convert the pendulum movement to the display of astronomical phenomena and the year according to the Gregorian calendar.

One of the interesting things about the Clock of the Long Now is that they tried lots of different designs for the mechanical computer, but ended up with digital mechanics for a lot of the same practical reasons digital electronics and digital information are so much handier than analog equivalents. And this is something that Hamming mentions here - that mechanical computers have been both "analog" and "digital" since the beginning - specifically Napier not only developed a logarithmic method of calculating, but his "Napier’s Bones" (ivory rods) were a digital calculation system.

Then we have Babbage in the 1820s, of course. And the Comptometer (1887) by Dorr Felt. I thought the whole Wikipedia entry for the Comptometer (wikipedia.org) was really interesting:

And the part about error detection and locking parts of the keyboard until errors were manually corrected.

There was a veritable explosion of these calculating machines in the late 1800s, and the Comptometer Wikipedia entry has a timeline with the dates.

Punch cards come from the mechanical computer/calculator era for storage of data and rudimentary programming. I’m a little surprised Hamming didn’t mention the Jacquard loom, which used punched cards in the 1700s (but, of course, you can’t mention everything in a short chapter!)

Side note: I once visited a textile museum in Rhode Island, where they had a working punchcard-driven loom that could produce woven bookmarks with letters that spelled out the museum name. I bought one and I’ve got that souvenir around here somewhere.

IBM (International Business Machines!) dominated in the early 1900s with punch cards and mechanical calculating machines.

Okay, I’d better shorten up these notes - I find all of this stuff super interesting, but if I dive into any of it, I’ll never finish my notes or this chapter: Zuse (electrical relays), ENIAC (vacuum tubes), etc.

Now Hamming puts up an interesting table with the comparative speed of operations per second. His table ends in the era in which he was writing the book - the early 1990s. He’s using operations per second, but we can probably substitute floating point operations per second (FLOPS) for any of the modern stats. From hand calculators (20 OPS) to 1990s digital computers (10^9 OPS/FLOPS).

And he makes a prediction that we can check (now that per-processor speeds have been slowing considerably):

Well, a factor of 100 on top of 10^9 would be 10^11, right?

So how did we do?

The Supercomputer (wikipedia.org) entry has us at 10^18 FLOPS. But it’s really not that simple because at some point we just started making supercomputers as huge clusters of Linux machines running on x86 (or ARM) CPUs, and various GPUs. To some degree, this is even pretty much just a ton of consumer-grade processing done in massive parallel. And a truly consumer-grade CPU, an Intel Core i5-9600K, can blast through 6.29 Giga-FLOPS, or 10^9 - which is Hamming’s supercomputer figure from 1990. Cool!

Anyway, I reckon we got a lot more than a "factor of 100" out of this architecture on the high end of parallel processing supercomputers.

This stuff is hard to compare, actually, but Hamming puts it well in the next bit, "the human dimension," when he writes:

Way to go little Core i5-9600K! I can buy one of those right now on eBay for $64. :-)

But, also:

To ensure everyone is "on the same page," so to speak, he reduces the modern design of computing hardware to: Storage (of instructions and data), Control Unit (gets and executes instructions), a Current Address Register, an ALU (does the actual calculating), and I/O for communicating with the outside world.

That is, the CPU is just a machine that operates on instructions. At this point, he gets slightly philosophical and makes vague threats about those looming "AI" chapters to come.

That’s it for Chapter 3. I’m going to have to be really careful about keeping my notes shorter because the next chapter is the software portion of computer history, one of my passions!

Let’s see if I can stay true to my word and keep this chapter’s notes short.

This is a great summary of the really early history of software development back to the days of programming in binary with absolute machine addresses by someone who was there while it was happening.

By the time I started programming computers, "spaghetti code" was used to mean "too many GOTOs". But Hamming clearly demonstrates how the original pasta was made by patching absolute memory addresses!

The lesson from this chapter is clearly to not resist obvious tooling improvements. Writing of programmers who clung to extremely laborious manual machine code programming versus languages like FORTRAN:

The thing is, I know I should be laughing at these old dinosaurs: "Ha ha, how silly! The fools couldn’t see how the computer could assist in programming computers!"

But here I am in the accursed year 2025 and it’s being seriously suggested that all programming (and every other creative endeavor, for that matter) should be completely turned over to so-called "AI" cloud services.

And I just can’t laugh at those poor old manual coding dinosaurs anymore. I feel…​deeply sympathetic for them.

Moving on, I totally agree with Hamming’s opinion about the proven success of human-oriented programming languages that are designed to help humans accomplish the task rather than stick with some sort of logical purity. Though I have yet to be convinced that a "natural language" angle has ever been a good way to bridge the gap!

I’ve never heard anyone express this sentiment about Algol before:

I thought this was really interesting and insightful:

That is, programming languages with high information density (low redundancy) are hard for humans to get correct: our natural language is very squishy, but highly redundant, so the signal gets through.

So how do you design a programming language with some redundancy to make the programmer’s intent clear? He mentions an anecdote where he approaches a language expert about the, "redundancy [we] should have for such languages":

I think that’s spot-on.

Also spot-on are these two conflicting ideas:

I do think comparing writing software with writing novels is a pretty decent comparison, though maybe not for the exact reasons he had in mind. (Source: I write software almost daily and have written two really terrible unpublished novels, ha ha.)

Hamming concludes this chapter on a sour note:

Ouch!

Hamming’s actual experience with the field of programming languages seems to be rooted in the FORTRAN era (1950s), with exposure to the Ada era (1980s). So I do take his opinions about the field with a grain of salt.

Hamming’s anecdote about how learning to give good presentations lead to having insights into the future of computing is really good.

He describes having to do extremely difficult mathematical work on barely capable early hardware to prove that computers could do work that wouldn’t otherwise feasible at all…​

I think this is an aspect of computers which is all too easy for those of us born after these machines have been at least somewhat commonplace: That their true value is to make things practical which would otherwise not be practical…​or even possible in a human lifetime.

A human could probably render a single frame of Donkey Kong by hand on paper. But a human could never render a single frame of, say, Doom 3.

And before we had Doom 3, we had long ago proven that we could do extremely difficult nuclear simulations on computers.

Here’s a heck of a quote:

Incremental improvement is great (awesome, actually), but I do have to agree that the initial idea has to pay off pretty quickly…​or it probably never will.

I’m struggling right now to think of notable exceptions to this!

Aside: Now I find myself thinking about Tim Berners-Lee being able to demonstrate his hypertext concept, the World Wide Web within about a year or two and it taking off like a rocket immediately afterward. By comparison, Ted Nelson’s similar Project Xanadu has been under continuous (I think?) development for 60 years and most people have never heard of it and probably never will. (Note: I’ve long been fascinated by Xanadu’s ideas, especially its advanced linking and the transclusion of virtual documents.) Actually, this particular cautionary tale has all sorts of lessons that would be totally ripe for a whole article of their own…​the value of ideas without an implementation, dreams versus pragmatism, "worse is better", etc.

Aside: And I really can’t help thinking about hype-driven "payoff" versus natural grass-roots "payoff". For example, Java had massive hype and was everywhere at one time and it did find a niche eventually (especially the JVM). But it did not take over the way it’d been hyped to do. In fact, and with great irony (given its marketing-driven name), I would argue that JavaScript is now the language of the universal machine that did take over, the Web browser, not Java and the JVM!

Ah jeez, now I’m rambling. Keep it short, keep it short.

Hamming, by his own admission, revisits Chapter 2 in recommending the use of general-purpose computers and software. But I take his point: this was once a much more controversial suggestion and one can extrapolate a lot of good ideas about what the future may hold if you trace a line from special-purpose calculating machines through the world of virtual machines and software of today. Where does that line lead in the future?

I’m so raw from the "AI" talk these days that I see the upcoming three "AI" chapters and I groan. But, of course, Hamming’s perspective on this from the 1990s should be fascinating.

What sort of things can computers do? Newell and Simon’s puzzle solving at RAND is something to look into. "Expert systems" are a term I remember from the 1990s, even though I wasn’t in the field.

Followed by this shot across the bow:

Ouch! Now do the study on engineers and mathematicians!

The impression I’m getting is that Hamming doesn’t have much respect for fields and abilities outside of his own. I would have fallen for this bait, hook, line, and sinker when I was younger.

He then claims that some rules-based systems have "shown great success". I’d be curious to hear an example. I’m not saying it doesn’t exist, I’d just like an example. Maybe that’s coming up?

Here, I think Hamming may have hit the nail on the head. I keep finding reasons to cite Gilbert Ryle’s work by way of Peter Naur, which is exactly about this question (and maybe Hamming was inspired by it?). Here’s what I wrote most recently: Go read Peter Naur’s "Programming as Theory Building" and then come back and tell me that LLMs can replace human programmers.

Another bang-on quote:

Yessss!!!!

I would go post this quote on Mastodon right now, but half of us are sick out of our minds from being bombarded by this subject non-stop from all sides and I try to limit how often I’m part of the problem.

Next is some fairly standard philosophy about what "thinking" really means and if it would ever be possible for a machine to "think". I think it’s a good summary.

I did give a dark chuckle at this:

Oh Hamming, if I could only transport you to the year 2025, when people are very impressed indeed by such statements.

The next (very) brief "AI" history and conceptual lesson is useful. And Hamming asks good questions at each step - none of which are really answerable in any definitive way!

I’m really curious where the next two chapters will take us…​

And there you have our current dilemma. (Well, one of them!)

I’ve been emailing someone smarter than myself who perfectly summed up the whole idea above as a question of "scale invariance". That is, if scaling a system up doesn’t change its properties, it’s scale invariant (fractals are a fun example). If LLMs are scale invariant, you’re not going to have intelligence pop into existance just by making them bigger.

(Although, we obviously can get systems that are capable of printing, "I am not scale invariant." See previous chapter notes.)

Then a discussion of the usefulness of computers and digital instruments in the field of music production, which I think makes some good (if simplistic) points. Then:

But he then has some pretty dismal things to say about computers competing with people anyway (for jobs) and the ability of your average "coal miner" to elevate to the types of work that a computer won’t replace.

I don’t think even Hamming could have foreseen that "the computers" would have come for the creative work before doing the menial work - because it turns out that’s easier to fake. :-(

He again stabs at the medical profession, suggesting that doctors could likely be replaced with expert systems eventually. I think he’s doing the classic engineer underestimation of the huge complexity of squishy real-world situations. I recognize it because I used to do it myself all the time and I know I was probably insufferable because of it.

To give Hamming credit, he mentions the legal problem: Who is to blame when a program makes a mistake? That’s an important question. But I don’t think he had the imagination or foresight in 1994 to ask the next question: Would you change your answers to any of the questions if you knew the organization using the program would do ANYTHING to make a short-term profit and would be delighted to find a way to shift blame away from the organization? (If my younger self could have heard me now. You have no idea.)

Oh Richard Wesley Hamming, my sweet summer child. If I could transport you thirty years forward to a hospital in this country now…​

But, of course, computers can be enormously useful, helpful, and wonderful. I wouldn’t love them if it weren’t so. But they must always, always, always be made to serve the user.

Computers can be helpful with calculus.

Some good practical points about robots!

He’s right about chess-playing computers eventually dominating. But this is a great quote:

Wow, Hamming is all over the place with this chapter, bouncing around from idea to idea with no clear connection. We’re back to philosophy.

This is a big WTF for me:

Is he saying that there’s crime in a particular neighborhood because the people who live there are…​just bad? I mean, I get the "free will" argument he’s going for, but I don’t think the argument makes any sense. Outside of horror movies, there aren’t just whole neighborhoods of bad people that exist for no external reason. (Again, if my younger self could hear me now…​ oh boy, what a shock.)

This paragraph near the end of the chapter has taken on a completely inverted meaning to me in 2025:

The inverse is just as true: "…​allows many people to believe machines can “think” since machines have produced the [appearance of the] required results."

Can’t argue with this:

This chapter is just a couple pages with a summary of the previous two in the form of a 9-point self-evaluation.

What I find fascinating is how much my personal feelings differ from what they would have been 10 years ago. Maybe even as little has 5 years ago? I would have been cheering all of this stuff on.

The machines aren’t the problem. Truly, they are just tools.

The problem, as ever, is who owns those tools and how those tools are used for or against people.

But I have to give Hamming full credit. The "machine learning" or "AI" field has turned out to be a big deal after all. And I don’t think that was obvious to everyone in 1994. I’m not 100% sure where the "AI" hype cycle was in the early 1990s, but I don’t think it was doing well? I imagine he was swimming upstream with these opinions and in the face of all the previous failed "AI" endeavors over the years.

Here’s some math. Hamming and the word "obvious":

Where am I going with this?

I have no idea.

I read every word of it, but I really don’t understand what’s going on here - and I don’t think it’s just my inability to divine insights from a page that looks like this:

I mean that’s also true. But I literally don’t understand the significance of the volume shrinking in relation to the surface area of a higher-dimensional sphere:

I took Hamming at his word in the introduction in which he wrote:

Is this chapter one of the "early parts"?

If I don’t respond to "obviously" with, "Yes, of course, Hamming, my good man, it’s a as clear as day!" does that mean I am meant to just skip it?

On one hand, I don’t think the mathematical insights in this chapter are translating very well to prose. It’s not that I don’t believe they exist, it’s just that I don’t think Hamming has succeeded in conveying them to me. And I think I did my part as a reader. On the other hand, I’m completely willing to accept that Hamming’s intended audience is, after all, post-graduate engineers and I’m not one of those. So maybe it will all be explained later and I’ll feel better about this.

(I also realize I’m outing myself as one of "those" programmers who doesn’t do some napkin calculus before writing a for loop. Yeah, sorry, guilty as charged.)

But you know what? I am curious how this higher-dimensional thinking will be applied later. I just hope I can understand it.

And, even better, he teased "Hamming distance". That sounded familiar. I looked it up and was pleasantly surprised learn that it’s a concept I am familiar with: edit distance (wikipedia.org). It just happens that the metric I’m used to is Levenshtein distance (1965). Since Hamming distance only allows substitution, it can only compare strings of the same length. Levenshtein allows deletions and substitutions - so it has no such limits for comparing the similarity of strings. Hamming introduced his distance metric in 1950.

I’m really looking forward to the information theory stuff in this book. It looks like that may begin in the next chapter?

Yay! Something I understand!

"Coding" in this chapter means encoding and decoding data for transmission or storage. And coding theory has its own Wikipedia page:

https://en.wikipedia.org/wiki/Coding_theory

It’s about the ways in which we can present a stream of data: in fixed or variable-length chunks and with or without prefixes. I feel quite at home with this subject right now since I’m working on an Emoji picker utility and am neck-deep in Unicode, UTF-8 encoding, and multi-character sequences.

Hamming discusses Kraft and McMillan:

https://en.wikipedia.org/wiki/Kraft%E2%80%93McMillan_inequality

Which…​uh…​yeah. I’m sure that’s very nice.

This page is helpful to explain what is being discussed, though:

https://en.wikipedia.org/wiki/Prefix_code

It just means that you can tell when a new "word" in the encoding is coming because the start sequence (prefix) can’t happen anywhere else (it isn’t valid inside of any other word.)

At an extreme end, a prefix of 0 marks the division of words in this binary sequence. The only thing we can encode are different quantities of 1 following the 0.

The Kraft sum is pretty cool, actually, since it tells you if your encoding is as efficient as it can be.

A perfect equality would be a binary encoding of five "words" like so:

Nothing is wasted. The first word is a single bit. And even though it doesn’t terminate with a 0, we know 1111 is word5 because there is no other valid sequence with four `1`s.

I’m curious where this is going in terms of the "Learning to Learn" subtitle’s premise.

TO BE CONTINUED