S3 Home‎ > ‎Educating the User‎ > ‎

Automated Calculation: A Brief History

Minimalist Summary

Calculation is a hard job. It requires extreme discipline and processes that are absolutely repeatable; things that humans are just not really good at. And so humans have been trying to automate calculation since the dawn of recorded history. This essay deals with the devices humans have used to automate calculation. It is in some sense a history of computer "hardware". 
There is a companion page on this web site that deals with what we might call "software", for lack a better term; algorithmic challenges of automated calculation, but this page deals with devices, ranging from clay tablets to quantum computers.

Historians would tell us that we can't understand the future without knowing something of the past. So the purpose here is to give some appreciation of a few major events in that computational history, especially in the context of modern data science and simulation science, which together make up computational science. And patterns are what make this venture worthwhile. Without patterns that repeat themselves and without those patterns dancing with other patterns, it's just stuff. The patterns are what make it history. The patterns are the things that allow us to look into the future by examining the past.

We are proud, at S3, to take a keen interest in the history of computation, so much so that we hold a substantial collection of older computational devices which we have acquired for the purpose of understanding the history of computation, and sharing that understanding with others. Most of the devices pictured here, particularly those associated with the Industrial and Information Age, are from our own collection.

Automated calculation is useful for lots of stuff. Throughout history, some of the biggest uses have been:

  • Engineering: Building bridges and buildings, transportation, etc.
  • Agriculture: Growing crops and feeding herds, and measuring the fields to do so. In fact the field of geometry--which is often compared to computer science--was originally developed by the Egyptians to measure growing fields.
  • Finance: Counting money, making money from money, comparing the worth of things, and doing so in a way that is reliable and accurate. We show an NCR model 442 cash register from our own collection at the right to honor the role money has had in advancing the hardware for automated calculation. It's no coincidence that the cash register arrived on the scene in 1883, and the adding machine arrived a year later. Once we had cash registers to hold the money we needed auditors to make sure it all added up. The NCR at the right is not quite that old. It hails from 1911.
  • Science: The language of science is mathematics, and if one is to verify one's science, one needs to make observations of the universe, apply the calculations of the science to those observations and then make more calculations to justify that the science describes the world as we observe it. Often, pre-modern scientists like Galileo or Kepler would need to make hundreds of thousands of calculations and so any means of automating them was welcomed. 

Of these, science is the odd man out, as the other three have a direct and immediate impact on commerce. In fact, Galileo developed the telescope to be the first to see merchant ships coming in to port as the first in line got the best deals. Once he saw the ships and profited from their arrivals, he turned his telescope to the heavens. Some of this remains true today. Telescopes in space were looking down at the Earth rather than up to the heavens thirty years before Hubble. 



Introduction

The story we are about to tell distinguishes "computation", which we define as automatic calculation, from other things modern computers do: things like archiving, searching, and communication/publication. We will mention those other things, but only briefly to contrast them with computation. The original Macintosh 128, for example, was extremely useful for preparing printed materials. It was said at the time that Gutenberg made everyone a reader, and Macintosh made everyone a publisher. That makes the original Macintosh hugely important. But the Macintosh 128 had little use in calculation. It came with a word processor and drawing program, but no compilers or programming language tools, and competent spreadsheets were not available for Macintosh until at least the following year, and only then with a memory enhancement. It was, in fact, rather difficult to program the early Macintosh even if you purchased those tools separately. Similarly, the Jacquard loom was an amazing computing machine that produced beautiful hard copy (woven silk fabrics) but was not very useful for calculation. Our focus here is on those things that caused quantum improvements in automatic calculation, not publication or communication or those other things. We also omit from our discussion those near misses that could have or should have been breakthroughs in practical computation. Charles Babbage's Difference Engine, for instance, could have created a revolution in computation. But it was largely of theoretical interest in the 19th century; historical interest in the 20th and forgotten in the 21st; a powerful idea that didn't really take hold. 

One of the amazing things about computing (as opposed to calculation) is just how general it is. The problems that gave rise to computer science have little to do with arithmetic or calculation. Alonzo Church, Alan Turing, and Emile Post, who are largely responsible for the development of computer science, were hunting much bigger game than calculation. They were concerned with "decidability": given an assertion of a logical truth, can that assertion be automatically verified. To give a little bit of context, the most interesting of these assertions deal with objects that look like computer programs. For instance, you may task me to write a calculation program that takes a cardinal number n as input and produces as output the nth prime number. Can you write a logical program that accepts whatever program I give you as input and decides whether my program meets its specification, returning "True" if it does and "False" otherwise? There are some obvious ways for me to write my calculation program. But your programming problem is not so obvious. It's a profoundly interesting topic, but one for another day. Our concern here is not about automating decidability (which cannot generally be done), but instead about automating calculation (which can be done).

When Gordon Moore described the growth in the calculating capacity of machines with his exponential "Moore's Law", he was thinking about the clock speeds of microprocessors; the "Integrated Circuits" stripe in the figure below, cribbed from Ray Kurzweil's The Singularity is Near, 2005. But Kurzweil views Moore's Law from a long term perspective that includes times prior to the rise of the microprocessor.




Basically, Kurzweil explains that the cost of computing (in inflation-corrected dollars) has been following a predictable and super-exponentially decreasing model since well before the microprocessor was invented. Venture capitalist Steve Jurvetson has said that this is possibly the most important chart ever produced. Jurvetson conjectures that this pattern may go back much much further and will continue much farther into the future. We will argue here that this particular pattern probably doesn't fit if you go back very far before 1900, although it will probably continue for some time into the future. We think Kurzweil's pattern is part of a bigger pattern characterized by long periods of relatively slow growth in calculation speed but punctuated by shorter periods of Kurzweilian exponential growth. We just happen to be in one of those Kurzweilian periods right now. The Kurzweilian periods involve the following events:
  1. A mathematical discovery is made. The mathematics is already there. Like the New World, it is discovered, not invented.
  2. A notation is invented to write down that discovery. The notation is no more the math than the word "skylark" is a bird. The notation is just one of possibly many ways of representing the math. Newton and Leibniz had a blood feud over different notations for the calculus. Often, the first notation we invent is inefficient. One way we get back on Kurzweil's curve is to invent a better notation. Roman numbers are one way to represent quantities. But the same quantities can be represented in Arabic numbers and that makes the quantities easier to apply and calculate with. Notation makes a big practical difference even if it doesn't change the math.
  3. Applications of the math allow us to design new things. Sometimes the math comes before the things. When Arthur Cayley discovered matrices, he thought they were pleasantly amusing. He had no idea that they would be the bedrock of electromagnetism and all of the invention of the modern age for that matter. On the other hand, sometimes the things come before the math. When we wanted to go to the moon, we knew what we wanted to build but we didn't have the control theory to build it. We needed to discover that so we set out on a focused search.
  4. Automatic calculation tools are invented to enable us to build those applications. Here again, confusion is possible. The abacus is not mathematics. It took the mathematics of number theory to motivate the abacus and the notation of a positional number system to build it, but the abacus is just an instrument to automate calculation. 
  5. A long period of slow refinement follows. And we have one last chance for confusion. The abacus didn't run out of gas. The mathematical discovery behind it did, and we waited for more mathematical discoveries.
Take the first example that we will discuss momentarily, from ancient Sumeria. The Sumerians:
  1. Discovered the formal concept of "quantity". 
  2. Invented a base-60 number system to write down those quantities.
  3. Realized that if you could work with quantities, that would enable things like economies and cities that we associate with civilization. 
  4. Invented the abacus to automate calculations with those quantities, and that made finance and engineering flourish.
  5. The abacus was gradually refined over a period of several thousand years.
While 1-4 were going on, growth probably followed a super-exponential pattern like Kurzweil shows. The period of slow refinement probably saw much slower growth in the ability to calculate. This is similar to logistic growth, where a population of bacteria will grow exponentially until it nears the carrying capacity of the Petri dish. Thereafter, growth slows. Then if something happens to increase the carrying capacity--like a better food source--exponential growth begins again until nearing the new carrying capacity. For the problem of automatic calculation, the carrying capacity is largely defined by the body of mathematical discovery. In the renaissance, that mathematical discovery was the logarithm (and the calculus that followed). In the Industrial Age the mathematical discovery is computability (and the various fields of computational complexity, numerical analysis, combinatorial optimization, etc. that are still shaking out). Notations like λ-calculus were developed to write down computability in a way where it can be communicated to others. We realized that we could use computability to build things. Computers were developed to perform those calculations, and now we are left to wonder whether we are still figuring out computability or if we are in the period of slow refinement.  



The Ancient World

Our story begins with a little junket to ancient Sumeria, first inhabited about 4500 B.C.E., and humans' first real go at civilization. And these Sumerians built stuff. What kind of stuff? Well, they're the ones that invented the wheel! And for that matter, they also invented the city. So they build stuff that moves around, and stuff that stands still. And when you build stuff you need to calculate, and therein lies a problem, because the number system that the Sumerians invented to conduct science and astronomy was base-60. Intellectual historians call this a "sexagesimal" number system. That seems like a bad way to go, but we can cut the Sumerians some slack because at the time, Europeans were still living in caves. Fortunately, the Sumerians also invented writing. Unfortunately, they did this writing on clay tablets which were very heavy and hard to carry around. A Sumerian student carrying in his backpack a copy of Moby Dick--or, more suited to the times, a preprint copy of The Epic of Gilgamesh--would need to be both a scholar and a champion weight lifter as those tablets would weigh hundreds of pounds. And nearly 1000 years before The Epic of Gilgamesh, or roughly 3000 B.C.E., the Sumerians wrote down their times tables on clay tablets so that they could remember all of the implications of this awkward sexagesimal number system. The image below, is one of a set of 13 such tablets from a 2010 exhibition at The Institute for Study of the Ancient World at New York University. These tablets could be thought of as the first known piece of mathematical software.


One could define a "Big Data" problem as one for which the size of the data set overwhelms established calculation methods. We actually don't believe that this is a good criteria for Big Data, and we explain why elsewhere, but it's a very useful criteria for understanding the history of automated calculation, and we will use it here. Prior to the Sumerian multiplication tables, any problem that required that two numbers be multiplied together would more than meet that definition of a Big Data problem. These tablets--while not exactly iPads--simplified that task, and made it possible to do other things, like discover algebra and geometry, and architect cities with plumbing and sewage systems, and pyramids and temples. And so we treat these Sumerian multiplication tables on clay tablets as the first human step in automated calculation. And for all of the inconvenience of this massive set of times tables, they still exist! And they will continue to exist long after the last DVD is no longer readable! Similarly, for all of the inconvenience of sexagesimal arithmetic, it remains with us as well, more than 5000 years later in quantities of time and angles (minutes:seconds).

But the Sumerians, not being content with mere times tables, went the extra step and invented the abacus somewhere between 2700 B.C.E. and 2300 B.C.E. The abacus can add, subtract, multiply, and divide. What we now call a 4-function calculator. The Sumerian or "dust" abacus was most likely made of more clay tablets covered with sand or dust, with words and letters drawn in the sand. The design of the Sumerian abacus would no doubt have been complex by comparison to later abaci as it had to support sexagesimal arithmetic. And since you need to know your times tables to use an abacus for multiplication and division, a person who calculated with an abacus in the third millennium B.C.E. would still have to carry around those nasty multiplication tables. Calculation was not for the weak in ancient Sumeria. But the abacus made possible another invention, that of money. If you build a city you need to have financial transactions. Doing lots of financial transactions you need a reliable way to add up numbers and the abacus helped with that. To this day, and for the foreseeable future, finance and accounting are still two of the most important areas in which automatic calculation is used.

To be sure, the Sumerian abacus was not very portable. But the idea was, and the trade routes were as much about transporting ideas as they were about transporting goods. After all, a good idea could make you a lot of that money stuff that the Sumerians invented. And in the second century of the Christian Era, the abacus had made its way via trade routes to China. This is about the same time that the base-10 "decimal" number system was developed in India. No doubt the decimal number system also made it to China on the same trade routes. Today, three primary kinds of abacus remain. These are:
  1. The Chinese abacus or suanpan, 
  2. The Japanese abacus or soroban,
  3. The Russian abacus, or schoty.
These three forms of the abacus resulted when the primitive bead abacus was combined with decimal numbers. There is a base-10 system lurking in each of these tools. The earliest reference to the suanpan is in Xu Yue's Supplementary Notes on the Art of Figures first published in 190 A.D. Every post--called "uprights"--has a total of seven beads on the suanpan. Here is a particularly nice antique suanpan:


The care and workmanship that went into these devices reflected their importance to their owners, and to society in general. A person who could calculate was esteemed, and the suanpan was essential to speedy calculation. The top regime of both the suanpan and the soroban is referred to Heaven, and the lower regime Earth. The suanpan has two beads in Heaven and five in Earth. This device can be used to represent decimal numbers in the standard Arabic number system. One would start with the rightmost upright (ones place). Each bead in Earth counts for one and each bead in Heaven counts for five. To add a number to the count, you would move the appropriate beads to the center divider, called the crosspiece. The configuration below, for instance, represents the number 7 (2x1 from Earth, added to 1x5 from Heaven).   


So each upright can be used for one decimal digit. The second column from the right is the tens place. So the following configuration represents 40 (4x10 added to 0 x 50). And so on.


Together the different columns can be used to count. Addition and subtraction are obvious applications of counting. Multiplication is a bit trickier. You need to know your times tables. You zero out the abacus (by moving all beads away from the crossbar) and then you add, place-by-place, by applying your times tables. For instance if you wanted to multiply 42 by 37, you would:
  1. Zero out the abacus,
  2. Multiply (7x2)x1 (in your head) and add 14 to the count, 
  3. Multiply (7x4)x10 and add 280 to the count,
  4. Multiply (30x2)x1 and add 60 to the count,
  5. Multiply (30x4)x10 and add 1200 to the count. 
  6. Read off the result, which is 1554.
Division can be performed in a similar but even more tedious manner (while the abacus is superb for addition and subtraction, it is less than ideal for multiplication and division). 

Later, the suanpan was imported into Japan, where refinements and improvements were made resulting in the Japanese abacus, or soroban, which has one bead in the heavenly regime and four (or sometimes five) in the earthly regime. The soroban is similar to the suanpan, except that the decimal places are read off differently; a slight change in notation. The rightmost upright is still the ones place, but instead of being able to represent 5 by either all of the Earth beads or one of the Heaven beads, it is represented only by one Heaven bead. Each upright still represents one decimal digit. The beads of the soroban are typically diamond-shaped (like two cones abutted bottom-to-bottom giving them an equator), as shown in the example below, an antique soroban from our collection. This diamond-shape is intended to allow the beads to be moved faster. 


In spite of its simpler design, the soroban proved faster for the expert to calculate with because the notation was more efficient. In fact, there were famous competitions in Japan in the 20th century where abacus operators would race against electronic calculator operators, and usually win. It is, however, more difficult to operate the soroban than the suanpan, requiring its user to think in terms of "complementary" numbers. Both the suanpan and the soroban are still used in certain parts of the world, though in the last generation, certifications have changed and Chinese accountants no longer need to pass proficiency exams with an abacus. 

The Russian abacus has only an earthly regime; no heavenly beads. The "uprights" on the Russian abacus are not really upright at all. Firstly, the abacus is laid on a flat surface so the rails run roughly horizontal. Secondly, the rails are not straight, they are bowed. Thirdly, the abacus is positioned so that the rails go from right to left. So for the image of the Russian abacus below (a specimen we acquired from Latvia in the former former Soviet Union) the user would be at the left of the image. By convention, the rail is zeroed when all beads are at the right, The abacus in the image below is zeroed. The beads are moved to the left for counting. Each of the bowed rails has ten beads, except, typically, one of the rails which has four beads. That rail is used for quarters, as in "a quarter of a kopek (a Russian unit of money)". Unlike the suanpan or to soroban, the Russian abacus is read with increasing magnitude from bottom to top (left to right in our image), with the quarters rail used to identify the decimal point. In the abacus of the image below, the third rail from the left in the image is the quarters bar or decimal point. The second rail from the left is the tenths and the leftmost rail is hundredths. The fourth rail from the left is the ones, the fifth tens, and so on. The Russian abacus has largely become an anachronism but may still be used by shopkeepers, On a 1997 visit to Russia, the researcher Ed Oswalt observed that "[t]he same store...where you can buy a Pentium computer, computes your bill using an abacus."


Wherever the abacus appears historically, scientific progress leaps forward: Sumeria, Mesopotamia, and Babylon, then Persia during the Sassanian Empire 2500 years ago, then Egypt and Greece a century or so later. The suanpan reaches China about 1800 years ago in the early Qin Dynasty, at a time when Chinese science flourished. The abacus reaches India roughly 1600 years ago, and then the soroban is imported to Japan from China in the 14th century. Up through 1990, Chinese accountants studied, and had to pass exams, on the abacus. The history of computation covers a period of 5000 years, and for the vast majority of those years, the abacus was the dominant form of hardware acceleration. 



The Renaissance 

Our discussion of the abacus has involved only arithmetic (4-function calculators). It is also curious that we never mentioned the abacus making its way to Europe. The Renaissance helps to take care of both those things. Europe was not the scientific or intellectual capital of the ancient world. It has been said that while in the Court of Charlemagne they were learning to write their names, in Baghdad they were exploring mathematics and charting the heavens. This is basically true. But the Renaissance changed all of that, particularly in the British Isles. In 1614, John Napier of Merchiston, Scotland, discovered the logarithm and published it in his tome Minifici Logarithmorum Canonis Descriptio. Napier is considered a mathematical genius, on par with Newton or Gauss, and logarithms are only one of his important contributions to calculation.


Logarithms give computational leverage. They allow multiplication and division to be expressed as addition and subtraction respectively, and they allow exponentiation to be expressed as multiplication. So,

a x b = 10log(a) + log(b)

ab=10b log(a)

We have yet to explain why exponentials are so important or why one would want to calculate them, and given that multiplication and division seem like more trouble with logarithms than without (because you have to calculate the logarithm first), it would seem like logarithms are going in the wrong direction. But as often happens, Napier was a mathematician who developed an idea whose practical consequence was only fully appreciated later.

Napier's work, published originally in Latin, was read by Henry Briggs, who visited Napier in the summer of the very next year. They became fast friends and Briggs moved in with Napier for a time and the two spent a month together working on tables for base-10 logarithms; what are now called "common" logarithms, the basis for the examples above. This, of course, required a great deal of calculation. By the (flawed) definition of Big Data that we offered earlier, in 1615, calculating the logarithm tables was decidedly a Big Data job. 

The year 1617 was a very important year for calculation, seeing three events of monumental importance to our story. As we said, the logarithm was only one of Napier's important works. The first great event of 1617 was Napier's invention--called Napier's Bones--to make the drudgery of calculating logarithms more manageable. Unlike the abacus, Napier's Bones is ideal for calculating the products and quotients of numbers.


Napier's bones are three-dimensional, square in cross section, with four different rods engraved on each one. A set of Napier's Bones usually includes a base board with a rim; the user places Napier's rods inside the rim to conduct multiplication or division. The board's left edge is divided into 9 squares, holding the numbers 1 to 9. The Napier's rods consist of strips of wood, metal or heavy cardboard. A set of such "bones" might be enclosed in a convenient carrying case, as with the set above, which dates from the late 17th century and is owned by the Science Museum in South Kensington, England. The second critical event of 1617 was Briggs' publication of the first base-10 logarithm tables in his Logarithmorum Chilias Prima, a brief 17 page privately published mathematical pamphlet (16 pages of which are the actual tables) making it possible for anyone who needed to calculate with logarithms to do so with only the modest tedium of looking them up. Briggs' tables were certainly more portable than Sumerian tablets, and astonishingly accurate. His calculations were to 14 decimal places and he erred only a few times and even then only in the last decimal place. He owed this accuracy, no doubt, to Napier's Bones.


The third important event of 1617 was the death of John Napier. One might say it was {Napier's Bones}-squared. Napier's Bones were used, and variations devised--sometimes in conjunction with an abacus for addition and subtraction--for at least 100 years, but shortly after their invention, an even more important one was made.

In 1622, William Oughtred, an Anglican minister, invented the slide rule (both circular and straight versions), which in its most fundamental form is two logarithmic scales that can be shifted to add the logarithms together. Where the abacus makes it trivial to add and subtract numbers, the slide rule makes it trivial to add and subtract the logarithms, which is how multiplication and division are done on a slide rule. Napier's Bones made it possible to calculate the logarithm tables, though with considerable effort. Then the logarithm tables made it possible to construct a slide rule. And then, since those tables are encoded directly on the scales of the slide rule, it is trivial to compute a logarithm or an exponent. The oldest known slide rule remaining intact was made by Robert Bissaker in 1654. 


Bissaker's slide rule is also in the Science Museum. It is about two feet long and has scales and slides on four sides. Most of the scales on Bissaker's slide rule are logarithmic or trigonometric. And it is definitely bulky. Not like Sumerian clay tablets but bulky. And we still haven't explained why anyone would really care about logarithms, especially with so many other things going on, like the plague, which was so paralyzing that Cambridge University was closed in 1665-1666. But it was exactly during this time of isolation that Issac Newton discovered what he described as the "Calculus of Infinitesimals" and what we now simply call the calculus. To be sure, the building blocks of calculus had been discovered by Fermat, Descartes, Pascal, and others. Newton's grand achievement was to unify what we now know as derivatives and integrals into a single holistic theory, where the derivative and integral are inverses of one another. This is the key mathematical discovery in making the logarithm and exponential so important to computation, because it leads to the discovery of the differential equation. The differential equation is an equation with derivatives. And the solution to the differential equation uses integrals. One of the most basic differential equations is:

y'(t) = y(t)

The solution to this differential equation is the exponential

y(t) = aet 

where e is the base of the "natural" logarithms (roughly 2.72), and a is some arbitrary constant. So there, finally, we have our exponential! As natural philosophers of the day began to explore the calculus, this differential equation or one similar to it showed up everywhere: in population growth, in chemical reactions, in diffusion, in finance, investment and banking. And wherever there was a differential equation an exponential needed to be calculated and wherever an exponential needed to be calculated a logarithm needed to be calculated, and wherever either needed to be calculated a slide rule was necessary. To appreciate the importance of these things, It's useful to remember that the natural exponential is also very closely related to the trigonometric functions and can be used to calculate the trigonometric functions. Euler's identity says that:

eit = cos(t) + i sin(t)

where i is the square root of -1. So the exponential describes stuff that diverges (et), stuff that converges (e-t), and stuff that repeats (eit). That makes it extraordinarily useful in the real world.

Note that there is no "cursor" (the sliding window with the hairline that indicates where the eye is to focus) in Bissaker's slide rule. Bissaker's slide rule lets you line up two scales. Since the slide has two edges, each slide allows two ways to line up two scales. And if there are lots of combinations of two scales that you want to line up, your slide rule will have to have a lot of slides. That's one reason Bissaker's slide rule is so bulky and four-sided. But Newton needed more. Newton needed to line up three parallel logarithmic scales in order to compute the roots of cubic equations. So Newton himself invented the slide rule cursor in 1675 to accomplish this. Newton's cursor failed to catch on at the time, but like Napier, Newton was a mathematician who developed an idea whose practical consequence was only fully appreciated later. The cursor on the modern computer screen is, of course, an extension of the slide rule cursor, and every time we follow the cursor on our screen, we owe a little bit to Newton, though probably not as much as we owe him for his differential equations, which underlie pretty much everything we know about the universe from the way patterns form on sea shells to the way securities on the stock market interact, to the way galaxies behave when they collide. To this day and for the foreseeable future, differential equations are one of the (if not the) most important sources of calculation. By our faulty definition of Big Data, solving differential equations was a Big Data problem at the time of Newton, and remains a Big Data problem today. One way we repay that debt to Newton is to use his name as the standard unit of force, and he was, indeed, an intellectual force.



The Industrial Age
 
The Industrial Age was a period where people were not afraid to build. They built bridges and dams and skyscrapers, they built cars and airplanes and ships that were cities at sea. Building requires calculation, and calculate they did! Engineers and architects often needed to calculate the area of a closed curve; a lake on a map, a plot of land, the cross section of a beam, etc. Green's theorem, from calculus, says that the area contained by a closed curve can be calculated by the line integral of the boundary of that curve. As early as the late 19th century to the mid-20th century, instruments called planimeters were made by Charles Bruning Ott, Eugene Dietzen, Keuffel & Esser, and others, that exploit Green's theorem to automate this calculation. The user could anchor one end of the planimeter to an arbitrary spot on the map and run the opposite end fully around the boundary of the region they wanted to measure, and then read off the contained area when finished. The planimeter pictured below is a Keuffel & Esser 4236, from our collection. This was one of the most popular models, and this specimen is a relatively early example, bearing the serial number 2328. To be sure, planimeters are still made. And they are conceptually not much different than this old K&E 4236. They are digital now, but work basically the same way they did in 1900.


For the accountant who performed mostly addition and subtraction, mechanical adding machines were developed by Victor, Burroughs, Monroe, Olivetti, and others. Many of those companies are still in business today and many still make business machines. Below is a Monroe model LX-160 adding machine from our own collection. The first one was manufactured c. 1920, and the last one c. 1960. Of the many models of mechanical adding machine, this one was compact enough to travel. The machine originally came in a black leatherette suitcase particularly well suited for auditors who spent most of their time at client sites.



Meanwhile, engineers and scientists calculated with logarithms, exponents, and trigonometric functions. The adding machine was not sufficient. They needed slide rules. Edwin Thacher, an American bridge engineer, patented a "Cylindrical Calculating Instrument" in 1881. The cylindrical design essentially folded the scales of a slide rule, whose combined length was over 30 feet. This allowed for calculations to between 4 and 5 decimal places of accuracy as needed by the science of the day. The instrument itself was produced by Keuffel & Esser as "Thatcher's Calculating Instrument model #4012" (note that a 't' was added in the middle of Thacher's name to make it agree with the common spelling). As with the abacus, these instruments were made with care and precision demonstrating both a pride of workmanship and a respect for the craft of calculation. A substantial fraction of the instruments manufactured still exist, now mostly in museum collections. The photo of Thatcher's Calculating Instrument below comes from a Skinner auction catalog. They sell for roughly $1000US at auction.


The engineer was, more often than not, at his drawing table. He set his slide rule on the table and used it in place. So these so called "Mannheim" slide rules had all of the scales on one side. They were typically made of fine wood, mostly mahogany, with a special white celluloid surface engraved with all of the necessary markings. Because they were on the drawing table, the Mannheim slide rules often had a supplemental ruler scale on the side so that they could be used to measure a drawing. Here is an example of a Keuffel & Esser Mannheim Polyphase slide rule from our collection, with the ruler scale facing the viewer. 


A very similar K&E Mannheim Polyphase plays a crucial thematic role in the film Titanic. As Titanic wanders, unaware, toward the iceberg April 14, 1912, the ship's chief architect, Thomas Andrews, ponders a blueprint at the drawing table in his stateroom. On top of the blueprint he has a book of calculations and a 10" K&E Polyphase slide rule that symbolizes his commitment to rationality. The slide rule case, a bit less weathered than the one in our collection, can be seen under a book at the left of the image below. Patented in 1900, the K&E Mannheim Polyphase was state-of-the-art at the time Titanic was built. It is well-known that the filmmakers went to great lengths to provide authentic scenery for Titanic, and the Mannheim was no doubt acquired after great deliberation, especially considering it's role at the critical turning point in the film. At the exact moment Andrews touches his slide rule (and to his great surprise), Titanic strikes the iceberg sending shudders through the ship! The image below, is captured just as Andrews extends his right hand to his Mannheim a moment before the collision.


Moments later, Andrews explains Titanic's fate to the executive crew. Titanic's owner, J. Bruce Ismay, asks how such a ship could possibly sink. Andrews replies that she is made of iron so she certainly can sink, and she will. He concludes: "it is a mathematical certainty!" The slide rule is a physical symbol of Thomas Andrews' mathematical certainty. But ashore, with the luxury of a larger drawing table, He would have preferred a longer slide rule for its improved accuracy. One of the best examples of this was the 21" Keuffel & Esser model #4096, shown below, also from our own collection.

 
The K&E 4096 was accurate enough to calculate ship parts and truss lengths for the manufacturing methods of the day. It stood on two cast iron legs (shown in the picture) to give it mass and stay put on the drawing table. A small knob at the left of the picture allowed the slide to be moved with the thumb. The foundation is mahogany, with a special white celluloid on the surface, etched with the numbers and markings. The back side of the rule had a table of conversion factors framed in beautiful finished mahogany. This particular model began manufacture in the 1930s, but a very similar 20" Mannheim Polyphase would have been available to Thomas Andrews. 

After World War II, the utterly civilized Mannheim slide rule was largely displaced by the more efficient "duplex" slide rule with scales and cursor on both sides of the rule. The picture below shows the "Post Versalog (top) and Keuffel & Esser Log Log Duplex Decitrig (bottom), both from our collection. The K&E shows a patent date of 1947. After the war, many engineers moved from the drawing table to the desk, and the duplex rule, held in both hands, was more convenient for a desk engineer. The Post and K&E are the slide rules that built the Boeing 707 and 747, the SR-71 (whose air speed records still stand after 60 years), and these slide rules sent the Apollo astronauts to the moon. In that sense, these slide rules may be the most important calculating instruments of the Industrial Age. The Post rule was somewhat more expensive, but built on a bamboo foundation, giving the rule a silky smooth glide in all weather. The mahogany-based K&E was believed to be a bit sticky in cold weather, but both are extremely fine instruments that were in manufacture and use up until the sudden end of the slide rule era in the mid-1970s.


To be sure, slide rules and electronic calculators did roam the Earth at the same time, although in the primordial days, electronic calculators were usually too big to roam very far. This 1952 National Geographic advertisement for the "IBM Electronic Calculator" (archived by the Slide Rule Museum) suggests that one of these devices compact enough to fit within a single room, could do the work of 150 engineers. To be sure, it could do the calculations of a team of engineers but it couldn't do the engineering of a team of engineers. While Neil Armstrong took his Post Versalog into space, the engineer front and center in the ad looking like Tom Hanks appears to be working the K&E Log Log Decitrig slide rule, as the distinctive lamination is evident in the larger image (click to see). 


But electronic machines like the IBM Electronic Calculator, while well-suited to certain high-throughput jobs, required special expertise and systems administrators, and experienced a great deal of downtime. Further, the engineer couldn't take the thing home and calculate at his private drawing table, or in front of his television. The IBM Electronic Calculator suffered from the same lack of portability that the Sumerian clay tablets suffered from 5000 years before. So the slide rule remained the engineer's primary tool for calculations for another 20 years after this ad was printed.

The electronic death blow was finally dealt to the slide rule not by some big space filling computer that was particularly fast, but by the Hewlett-Packard HP-35 hand calculator, which was particularly portable. Introduced in 1972 and so named because of the 35 keys on its surface, the HP-35 needed only to be faster than moving the slide on a high quality slide rule, and do log and trig functions. The earlier 4-function calculator was good for accounting, and small electronic versions had been available since at least 1968. But it was the ability to compute logarithms and trigonometric functions that had made the slide rule indispensable and the HP-35 could do those calculations more quickly, with fewer human errors and greater accuracy. The transition came fast. So quickly and so completely in fact that brand new, in-the-box Post Versalog slide rules made in 1974 are still available today! They were manufactured, sent to shops, and the market dried up before they were sold. The German company, Faber-Castell also still has an inventory of unsold slide rules and ones can be purchased by contacting the company. In spite of the diversity of products they offered, Keuffel & Esser relied so much on the slide rule for their base business that they filed for bankruptcy in 1982, just a few years after the electronic calculator revolution. In 1965, the engineer of refined taste wore a Post Versalog prominently on his belt. In the 1975, the engineer of refined taste wore an HP-35 on his belt.

None of the 35 keys on the HP-35 are "=" or parentheses or braces. The HP-35 introduced the still popular Reverse Polish Notation (aka Polish Postfix or RPN) to the engineer and scientist. RPN came from the study of logic (the Lisp programming language uses a similar logic). It allowed complex calculations to be specified without use of those keys, making for fewer keystrokes than algebraic notation. It's another case where notation makes all the difference. RPN distinguished Hewlett-Packard's brand from competitors, especially Texas Instruments, which began producing scientific calculators soon after the HP-35 proved successful. The Texas Instruments calculators usually bested the Hewlett-Packards in feature count, and were less expensive, but users preferred HP because of RPN. 


The HP-35 cost about ten times as much as a slide rule, but engineers rushed to get theirs anyway, giving Hewlett-Packard a commanding market share and allowing them to sell subsequent calculators like the richer featured HP-45, at high price. Hewlett-Packard's success in hand calculators led to the announcement, in 1974, of another revolutionary improvement in the ability to calculate automatically. The HP-65 not only supported slide rule-like functions, it also had 5 programmable keys (20 if you include the blue, yellow, and inverse-yellow "shift" functions), and a magnetic card reader that could be used to read and write programs of up to 100 operations. And those programs could also have logic, jumps, and loops, so a program could potentially replace thousands of calculator keystrokes in fixed-point operations like root finding, or in computing convergent series. To be sure, there had been earlier programmable calculators, going back at least to the Casio AL-1000 from 1967. But those machines were big and bulky, difficult to program, and rather delicate. They never really took hold. The HP-65 did. 

The HP-65's card reader created a secondary market for programs in specialized areas like finance, electrical engineering, surveying, probability and statistics, navigation, etc. The person who owned an HP-65 could take advantage of its programmability without ever having to write a program. It was a most remarkable device and everyone wanted one. At $695, not everyone could afford one, but engineering groups would often buy an HP-65 and loan it out. Engineers could take it home for the weekend in its sturdy case with, say, the structural engineering software package. The availability of a department HP-65, and the fun of using one, got a lot of engineers to do unpaid work on their weekends.

   

When Texas Instruments offered a card reader on their less expensive TI-59, they gained market share, and in 1976, Hewlett-Packard offered an upgraded version of the HP-65, labeled the HP-67, at $495. Still more expensive than the TI, these were an instant hit. Calculators from HP and TI competed and soon supported virtually any built-in function that could be desired, software modules available as plug-in chips, optical wands to read programs in barcode form from the magazines that the manufacturers published for their loyal customers, etc.  

But there were lots of problems that were bigger than calculators, and computers were getting smaller, cheaper, and more reliable. The really big computers, like the IBM 360 series were typically reserved for business purposes, focused less on calculation and more on information storage and retrieval. And scientists who were forecasting the weather, or designing more crash-resistant cars, or solving big optimization problems needed computer time. The mini-computer fit the bill. A department or engineering group could bring in a mini-computer like the DEC PDP-11, (pictured below, c. 1975). These computers were reliable, and less expensive than the IBM mainframes making them ideal for people who needed to calculate before the rise of the personal computer. The PDP-11 became ubiquitous. It was the standard machine students learned to program and the standard machine professionals used to calculate. The PDP-11 is still in service in some places, decades after the manufacturer, Digital Equipment Corporation, went out of business. In fact, Canada plans to use the PDP-11 to control certain power plants until 2050 (the wisdom of that decision is probably suspect). 


In the 1970s, spreadsheets were common in business. And by spreadsheet we mean a wide piece of paper with rows and columns of data and calculations, just as the name suggests. Ledger paper with all of these rows and columns marked with fine lines was available from any good stationary store. But these paper spreadsheets were error prone. The errors propagated easily and corrections propagated with considerable difficulty. In 1979, big spreadsheets were a big data problem by our previous definition. Mitch Kapor, a student at MIT's Sloan School of Management, decided that his homework was harder than it needed to be, and so he inspired friends Dan Bricklin and Bob Frankston to program the first electronic spreadsheet for his Apple II personal computer to make homework easier. By all indications, the three were all good students. Bricklin and Frankston finished their degrees but Kapor quickly abandoned his education to work on the spreadsheet program. Bricklin and Frankston went on to form Software Arts, the company that developed VisiCalc (the first electronic spreadsheet program for desktop computers), and Kapor went on to found VisiCorp, which published VisiCalc as "The Visual Calculator" in 1979. Kapor may not have made his degree but he did make a whole lot of that money stuff that the Sumerians invented. 

VisiCalc was originally available only for the Apple II computer, which had only 40 characters of screen width (as per the image below). The utility of VisiCalc created a large secondary market for 80 column cards that could be installed in the Apple II to give twice as many columns on screen. For that matter, VisiCalc created a large market for Apple II computers. Prior to VisiCalc, desktop computers were really only useful to people who programmed them, at least for purposes of calculation. VisiCalc and Apple II were a bit like the HP-65. They gave the programmable desktop computer relevance for the non-programmer. The most commonly used features of modern spreadsheets were already present in VisiCalc (which had to be small to operate in 64KB of memory), and the spreadsheet programs that followed, like MultiPlan, Lotus 1-2-3 (another Mitch Kapor invention), and Excel (originally in 1985), largely mimicked VisiCalc while correcting some of its notational idiosyncrasies. Once again, notation made a difference, and Lotus's more convenient notation led to the eventual extinction of VisiCalc.


Mitch Kapor, incidentally, is still around. He made enough of that Sumerian money stuff at VisiCorp and Lotus to become a philanthropist and all around good guy. At the time of this writing, Mitch is a principal at the Electronic Frontier Foundation, which provides funds for legal defense in court, presents amicus briefs, defends individuals and new technologies from what it considers abusive legal threats, works to expose government malfeasance, provides guidance to the government and courts, organizes political action and mass mailings, supports some new technologies which it believes preserve personal freedoms and online civil liberties, maintains a database and web sites of related news and information, monitors and challenges potential legislation that it believes would infringe on personal liberties and fair use, and solicits a list of what it considers patent abuses and works to defeat those patents that it considers without merit. The Electronic Frontier Foundation has been very vocal in their advocacy for net neutrality, and influenced recent FCC decisions that largely preserved net neutrality. So the next time you download that high resolution GIF file in real time with confidence that your connection is private and secure, you owe a little bit of gratitude to some MIT Sloan School professor who made Mitch's homework a little too hard.

At roughly the same time that personal computers were liberating people to do relatively big calculation jobs without needing a sysadmin, another set of really really big calculation needs was developing that the personal computer, and the mini-computer, and even the mainframe computer couldn't tackle. The problems had to do with things like simulating nuclear blasts or hurricanes, or the flow of an aerosol from a high pressure nozzle. These problems required a "supercomputer". The Cray-1, pictured below (c. 1980, from the Deutsches Museum), was capable of 160 million floating point operations per second (MegaFLOPS), and supported 4MB of RAM, for a total cost of about $9 million. 


The Cray-1 was the brainchild of Seymour Cray, whose whole life was devoted to building the world's fastest computers. This class of supercomputer reigned supreme in calculation for more than a decade. Of course other Crays, with better speed and more memory, were to come, but the basic idea of a special supercomputer processor was state-of-the-art and a central tenet in Seymour Cray's vision of the supercomputer for his entire career. Engineering workstations and even personal computers would eventually reach the same level, but they were a few years behind, and in those between years, problems could be solved on the "big iron", as well as establishing marks for the workstation market to meet. With this power also came possibilities for new software. And it is no coincidence that the first computer algebra system required about the same computational power that the Cray-1 offered. The Mathematica program, from Wolfram Research, was the first widely used computer algebra system and is still the leader. When it was introduced in 1984, engineering workstations had already caught up to the original Cray-1. Personal computers had not quite done so, and while Mathematica could run on a personal computer, that computer would have to be hotrodded with at least 4MB of RAM, which was a considerable penalty since the operating system was designed for a maximum of 1MB. So what did Mathematica offer that convinced people to violate the warranties on their desktop computers?



There were other programs that did arithmetic, but Mathematica did mathematics. It could take a complex expression and simplify it so that the minimum number of calculations were necessary. It could take a formula that was sensitive to roundoff error and make it stable (digital computers whether they are supercomputers or laptops have a hard time multiplying big numbers by small numbers, and this leads to roundoff errors that undermine the integrity of the results). Basically, Mathematica made it possible for people who calculate to do the right calculations. In a sense, Mathematica is where calculatability and decidability meet. So it is no surprise that computer algebra systems later became a part of handheld electronic calculators like the Texas Instruments TI-nSpire line. Mathematica is still going strong, and its distributor, Wolfram Research, proudly proclaims that no fewer than 10 Nobelists in physics have personally registered copies of Mathematica, and that the famed mathematician John Forbes Nash (the subject of the movie A Beautiful Mind and a Nobelist in economics) was one of the best-known Mathematica users.

The cost of a Cray supercomputer being a nonstarter for most organizations, even well-funded organizations like NASA, and the cost per FLOP for a commodity computer diminishing rapidly combined to bring about the curiously named "Beowulf cluster". Beowulf was a modern take on the Butterfly Machine that BBN had built a decade earlier. BBN had built the first computer networks and the fastest computer networks, and their engineers thought that a superpowerful parallel computer could be built by using a fast ethernet network as a backplane. While the Butterfly Machine was a famous proof-of-concept, its component parts were fairly expensive: Networks were expensive in the 1980s and the computers Butterfly connected were relatively expensive. But by the time of Beowulf, 100-base-T networks were cheap, IBM PC commodity clones were cheap, and the Linux operating system was free. A Beowulf cluster is basically a Butterfly Machine made from commodity computers running Linux and networked via a high-speed commodity network. The first Beowulf was developed at NASA Goddard in the mid-1990s. These clusters could give researchers the FLOP power of a supercomputer but at a fraction of the price. They required that the software be able to spread the job over a large number of processors, but if that could be done, these commodity computers en masse could deliver superb value.


Seymour Cray never bought into this idea of parallelism in high-performance computing. He once told his naysayers: "If you were plowing a field, which would you rather use: Two strong oxen or 1024 chickens?" But eventually Beowulf-style clusters out-calculated even the fastest Cray, and the supercomputer transformed itself to become a Beowulf cluster...on steroids. This was good in big ways. This kind of supercomputer is easier to extend. You can add new processors, and so it has a longer useful life. When parts go bad, they are replaceable. The software that runs on a desktop computer can also run on the Beowulf. The advantages won out, Cray sales dried up and the market for Cray supercomputers disappeared almost as fast as the market for Post Versalog slide rules had disappeared 20 years earlier. Cray filed for bankruptcy in 1995. The next year, Seymour Cray died from injuries sustained in a car accident. In the wake of Cray's death, a large number of circuit boards in his personal archives were nicely framed and sold as commemoratives, the proceeds going to support Seymour Cray's vision of faster and faster computers. The memory board in the image below is from a Cray-1, serial number #263, from our collection. Today, it is a fabulous souvenir of a bygone day of supercomputing, though useless in almost every other way. It's the new Mannheim slide rule.
          

But at some point strapping more commodity computers together ceases to scale. You spend more time spreading calculations out across the processors than you spend doing calculations. Cray's thousand chickens argument held a grain (or two) of truth. The special purpose supercomputer still had some major advantages. Mainly, things were close together in the Cray so it was fast to move data from one place to another. If you need to move a piece of data to a processor across the room, that may take one-ten-millionth of a second at the speed of light. That may seem fast, but at gigahertz clock speeds that's 100 clock cycles wasted. Cray's approach put everything close together so that the speed of light was less of an impediment. What was needed was something like Cray's special purpose superfast processor, but built at commodity scales so that prices were as low as possible.

Fortunately, there are enough people who play computer games and require computers to satisfy the special purpose of making calculations to draw things on the screen that these special purpose computers can be mass produced and sold relatively inexpensively. They are called Graphics Processing Units or GPUs. Their job is not to manage program logic, or retrieve sales receipts from a database. Their job is to calculate! The GPU has been re-purposed for scientific calculations with outstanding success, and is the main reason that we are still on Kurzweil's extended Moore's Law curve. And GPUs can be ganged so that several of them can be working at the same time. Instead of plowing with 1000 chickens, GPUs let you plow with 1000 oxen. GPUs have lots and lots of parallel processing cores. Compare a high-end server processor like the Intel Xeon with its 8 parallel processing cores to a GPU like the nVidia TITAN X with its more than 3000 parallel processing cores. This is still calculating with parallelism, but the difference is that these processing cores are in the same place. If the parallel processors are a few microns from one another, it doesn't take much time to move the data around. They can behave like one unit. For purposes of calculation, this is kind of the best of both worlds.

The image below pictures four nVidia TITAN GPUs. Each sells for about $1000, and is capable of about 7 trillion floating point operations per second (TeraFLOPS). So each of the TITAN GPUs in the image has the calculating power of about 43,000 Cray-1 supercomputers. Together, this bank of GPUs has the calculating power of 175,000 Cray-1 supercomputers, and at a total cost of about one-two-thousandths of the Cray-1. That's a cost per FLOP advantage of about 350-million-to-1. This configuration is becoming standard for applications in image understanding and natural language processing. In fact, the image below is an nVidia DEVBOX, which is a Linux workstation with a single good quality CPU and 4 TITANs. Basically, when we want computers to "think" the way humans think, we simulate human processes using huge numbers of calculations, and in 2016, calculation of that magnitude is performed on GPUs.

https://developer.nvidia.com/sites/default/files/akamai/cuda/images/product_supporting_images/nvidia-digits-side.jpg

None of this progress is lost on supercomputers, and the modern supercomputer is like a Beowulf design with commodity processors like the Intel Xeon, combined with GPUs like the nVidia Tesla, and special purpose processors like Altera Field Programmable Gate Arrays. China's National University of Defense Technology currently manages the world's most powerful supercomputer, called Tianhe-2, or "Sky River 2". This supercomputer combines 32,000 Intel Xeon processors with 48,000 GPU-like processors from Intel (Xeon Phi--the prior version, Sky River I used nVidia Tesla GPUs). Sky River 2 achieves 34 thousand trillion floating point operations per second. That's the equivalent of 212 million times the calculating power of Cray-1!


   


What's Next?

If the past is our guide to the future, we need to look at the the pattern, which calls on us to start with mathematical discovery. There are some fundamental problems facing people who calculate, and these problems are knitted into complexity theory, which is itself knitted into the fabric of the universe. Let's state a classic problem to illustrate this. Since computers play a big part in this story, let's say that we need to drill holes in a computer circuit board and the net cost of drilling the holes is directly proportional to the distance that the drilling head has to move. So we want to calculate the shortest path that the drilling head needs to travel to drill all of the holes. This problem is in a complexity class known as NP-Complete. This means that the number of elementary calculations required to compute the best path grows as an exponential function of the number of holes to drill. So if it takes an hour to compute a best path for a 100-hole circuit board, it might take a year to compute a best path for a 200-hole circuit board and from now until the end of the universe to compute a best path for a 300-hole circuit board. And it turns out that this complexity is burned into the fundamental mathematics of the problem. So the reason that the time grows exponentially is not due to us using a bad way to calculate. Any way to do the calculation would have the same problem. To be fair, we haven't discovered all of the math needed to prove that, but unless there is suddenly a highly unexpected mathematical discovery, that's just the way the universe is! This is one reason why we need bigger and bigger computers. Let's say that we could compute the the best drill head path for 100 holes on a Cray-1. Now we move that program to Sky River 2. We might be able to do a 200-hole problem. Suddenly, that 200-million-to-1 boost doesn't seem that big because all of that progress gets absorbed in the underlying complexity of the universe. Another way to look at it is that if the number of calculations is growing at an exponential rate and the computational complexity of the problem is also exponential, then the size of the problems we can solve is growing only linearly. If it took 35 years to get from 0 holes to 100 holes, then we should expect it to take another 35 years to get from 100 holes to 200, and a very long time to get to 1000 holes. By the standards of our earlier definition, NP-Complete problems are Big Data.

The quantum computers that get a lot of ink in futuristic magazines won't help with the circuit board drilling problem. Quantum computers don't beat NP-completeness. The glossy magazines often say otherwise, but they are wrong! Umesh Vazirani, a Berkeley professor of extraordinary ability proved this a quarter century ago, and Vazirani's result was verified by other really smart people, and is well-known to people who care about this stuff. Quantum computers do help with a different class of problems that are not quite NP-Complete. That class is called BQP (bounded error quantum polynomial time). BQP and NP-Complete are distinct classes. No problem in BQP is also in NP-Complete. Most study of BQP focuses on cryptographic encoding, making it a very interesting line of study. There are at least two well-studied calculation problem in BQP. One is Integer Factorization and the other is the so called Discrete Logarithm problem, which as the name suggests is the integer version of Napier's logarithms. and as Napier's logarithms were very hard to calculate when he discovered them, discrete logarithms are very hard to calculate without a quantum computer. Quantum computers already exist in one form with the D-Wave Adiabatic Quantum Computer or AQC. So why isn't this part of the story in the past rather than in the future?


Well, the AQC is sort of a low-grade, nickel-plated quantum computer. It is not as powerful as the Circuit Quantum Computer (CQC), which is years away. As an example of how far we are from the quantum payoff, the largest nontrivial integer that has been factored on an Adiabatic Quantum Computer (the gigantic, multi-million dollar D-Wave 2 in the picture above) is 56153. Mathematica factors that integer on a notebook computer in 12 microseconds (the answer is 56153=233x241). In fact, Mathematica on a notebook computer factors the number 57249139981806679451 in less than 4 milliseconds (the answer is 57249139981806679451=2543568463x22507410677, like 56153, this number is also a product of two primes, which is our criteria for "nontrivial"). A CQC would factor either of those numbers effectively instantaneously. The Circuit Quantum Computer may end up being the 21st century's version of Napier's Bones. Or it may end up being the 21st century's version of Babbage's Difference Engine and relegated to the role of historical curiosity.

There is another interesting corner case of problems that are almost in the class NP-Complete, but not quite. And some very compelling mathematics has been developed to exploit those corner cases, giving a dramatic reduction in the number of calculations needed to solve a certain class of "Curse-of-Dimensionality" problems. To return to the pattern that we started with, we have discovered the math to convert these Curse-of-Dimensionality problems into another class called Curse-of-Complexity problems, and and we have the math to solve some of these Curse-of-Complexity problems without the exponential explosion. We even have some notation to write all of this stuff down. In all likelihood it's not the best notation, because it is crazy-difficult to read and absorb. But at least we have a notation. We also know that if we do these computations, we can solve a very useful class of problem. Right now, pretty much all of those problems under study are control theoretic. Much of the mathematics and notation is captured in William "Mac" McEneaney's Max-Plus Methods for Nonlinear Control and Estimation, also known as "The Green Monster". The Green Monster isn't a big book, but like Napier's Logarithmorum, it is formidable.


None of this may seem all that useful if you're not a control theorist, and at the time of this writing, The Green Monster is ranked 2,562,935 in Amazon books. Probably about where Napier's Logarithmorum would have been before the idea caught on. But at this particular point in history, all of this is profoundly useful because we've passed the point where the number of processors exceeds the number of people. If you have one computer per city, like we did in 1950, then it's no problem to have 100 sysadmins to take care of that computer, to feed it data, and act on the results. When we have one processor per person, that person also has to manage the care and feeding of their processor. They can still sysadmin the system, but it's different because they can't be full time sysadmins because they still have to get real work done. But when we have 100 processors per person, the processors need to sysadmin themselves. They can't have humans collecting the data for them; they can't have humans handling the results. They have to do that themselves. The computers need to (metaphorically) press their own buttons and read their own screens). The humans can tell them what their goal is, but then the processors need to take it from there. That's a job for control theory. One of the kinds of control theoretic problems this "Curse-of-Dimensionality Free" mathematics is good for is the kind of problem where you need to control a system in a real world setting, i.e. you don't have all of the information you want, and other systems are not only competing with you, but they can affect the messages that your sensors see and your controls act on. For an example problem, consider driverless cars. Right now, driverless cars are kind of like computers in 1950; there aren't a whole lot of them and they seldom compete with each other on the road. But when all cars are driverless, it will be different. One car could, for instance, change lanes to give your car (and all the cars around you) evidence that traffic patterns up ahead are different than they really are, causing your car to react by heading straight into a traffic jam, leaving a less congested path for the other car. These driverless cars are sneaky. And their control systems need to be competent at dealing with their sneaky brethren. 

We don't really have special calculation tools for doing Curse-of-Dimensionality-Free calculation yet. But we expect this, and things like this to fuel automated calculation in the not-so-distant future. One possible way this could happen is through a new processor architecture being explored by Intel. A processor's speed is typically measured by its ability to do arithmetic calculations. That's a big part of what Moore's Law represents. And so processors optimize for this arithmetic, what is called the "standard ring", where a ring is an object in an abstract algebra. The standard ring has standard addition and multiplication operating on floating point numbers, but rings can involve all kinds of things. We could, for instance, have rings of matrices, or rings of images with their associated operations. Curse-of-Dimensionality Free methods rely on a different kind of arithmetic. They use something called idempotent algebras, which involve different rings (or semi-rings) over the real numbers. The key algebras are Max-Plus and Min-Plus algebras. Computer processors aren't optimized for Max-Plus algebra. Which means that while they may speed the process up considerably, there is still a lot of room for improvement if we were to add some Max-Plus instructions to the processor. That isn't happening any time soon. But what is happening is that Intel is adding a Field Programmable Gate Array (FPGA) to their Xeon processor. We discuss this in a white paper elsewhere on this site. The FPGA is a customizable piece of hardware. In essence, the FPGA gives you the ability to add your own custom instructions to the processor's native instruction set. And it will be right on the processor, making it fast. Max-Plus algebra could become part of native instruction set through the on-chip FPGA. This would be one way to improve automatic calculation for Curse-of-Dimensionality free methods. 

The Curse-of-Dimensionality-Free calculation, and Quantum Circuit Computers are a couple of the upcoming events that we think will change automated calculation. In the slightly more distant future we may have implants to automate calculation. The pattern of history suggests that portability is really important, and often more important than speed of calculation. A brain implant would make calculation about as portable as possible. 

It is already possible, in the lab, for people to use their brain activity to control artificial limbs, and devices like BrainPort (pictured below) exist to allow the blind to see through optical sensors connected to their tongues. 



The ability to control artificial calculators is likewise plausible: think of a calculation and it is done. For most of history, we have used machines to do things that humans aren't good at. Robotics, and things like driverless cars, have made it possible for machines to do things that people are good at. Our ultimate revenge and vindication may be to employ brain implants so that humans can do things that machines are good at.


pS3 Data Science
S3 Data Science, copyright 2016.