Course Handout - Short-Term Memory Subtypes in Computing and Artificial Intelligence

See More

Part 4 - A Brief History of Computing Technology, 1951 to 1958

Copyright Notice: This material was written and published in Wales by Derek J. Smith (Chartered Engineer). It forms part of a multifile e-learning resource, and subject only to acknowledging Derek J. Smith's rights under international copyright law to be identified as author may be freely downloaded and printed off in single complete copies solely for the purposes of private study and/or review. Commercial exploitation rights are reserved. The remote hyperlinks have been selected for the academic appropriacy of their contents; they were free of offensive and litigious content when selected, and will be periodically checked to have remained so. Copyright � 2010, High Tower Consultants Limited.

�

First published online 10:30 BST 17th July 2003, Copyright Derek J. Smith (Chartered Engineer). This version [HT.1 - transfer of copyright] dated 18:00 14th January 2010

�

�

This is the fourth part of a seven-part review of how successfully the psychological study of biological short-term memory (STM) has incorporated the full range of concepts and metaphors available to it from the computing industry. The seven parts are as follows: � Part 1: An optional introductory and reference resource on the history of computing technology to 1924. This introduced some of the vocabulary necessary for Parts 6 and 7. To go back to Part 1, click here. � Part 2: An optional introductory and reference resource on the history of computing technology from 1925 to 1942. This further introduced the vocabulary necessary for Parts 6 and 7. To go back to Part 2, click here. � Part 3: An optional introductory and reference resource on the history of computing technology from 1943 to 1950. This further introduced the vocabulary necessary for Parts 6 and 7. In so doing, it referred out to three large subfiles reviewing the history of codes and ciphers, and another subfile giving the detailed layout of a typical computer of 1950 vintage. To go back to Part 3, click here. � Part 4: An optional introductory and reference resource on the history of computing technology from 1951 to 1958. This material follows below and further introduces the vocabulary necessary for Parts 6 and 7. The main sections are: � 1 - Computing 1951-1958 - First Generation Hardware (US) 2 - Computing 1951-1958 - First Generation Hardware (UK) 3 - First Generation Software 4 - First Generation Artificial Intelligence � Part 5: An optional introductory and reference resource on the history of computing technology from 1959 to date. This material will further introduce the vocabulary necessary for Parts 6 and 7. To go directly to Part 5, click here. � Part 6: A review of the memory subtypes used in computing. To go directly to Part 6, click here. � Part 7: A comparative review of the penetration (or lack thereof) of those memory subtypes into psychological theory. UNDER CONSTRUCTION. � To avoid needless duplication, the references for all seven parts are combined in the menu file. To return to the menu file, click here, and to see the author's homepage, click here. We also strongly recommend the computer history sites maintained corporately by the Charles Babbage Institute at the University of Minnesota, the Bletchley Park Museum, the Computer Conservation Society, and the National Archive for the History of Computing at Manchester University, as well as those maintained individually by the University of Essex's Simon Lavington, Clemson University's Mark Smotherman, and the freelance historians George Gray (editor of the Unisys Newsletter) and Ed Thelen.

�

1 - Computing 1951-1958 - First Generation Hardware (US)

After the top secret development effort of the 1940s [see Part 3], a number of civilian General Purpose Computers (GPCs) were delivered more or less simultaneously in the early 1950s, led by the Ferranti Mark I and the LEO I in February 1951, and followed closely by the US Census Bureau's UNIVAC in March 1951. The design principles were those of the 1950 Eckert-von Neumann machine [reminder], and the technology was basically chassis-mounted valve-based circuitry - hot, heavy, and unreliable. The stage was then set for a relentless expansion in commercial data processing capacity as large corporations started to install Eckert-von Neumann derivatives - known nowadays as "First Generation" Computers - alongside their existing data tabulating machinery. As the decade progressed, a number of other GPCs emerged from the established centres of excellence, and the final list includes the IBM 701, the Harvard Mark IV, the Bell Labs Models V and VI, the NPL's DEUCE, Cambridge's EDSAC II, Manchester's Marks II through V, and the LEOs II and III; each faster, more reliable, and offering more memory than its series predecessor. But because the centres of excellence were themselves still learning, some surprisingly fundamental innovations were still being made, and in due course these would usher in a newer generation of systems. The transistor was one such innovation, and, having invented the technology in the first place, Bell Labs were one of the first to experiment with transistorised digital circuitry, and their 1954 TRADIC contained 700 point-contact germanium transistors. Manchester University did likewise with two small research machines, which they had running in November 1953 and April 1955 respectively, and so, too, did the UK Atomic Energy Research Establishment, Harwell, with their 1955 CADET. Among the lesser innovations in this period was the development of magnetic tape storage .....

�

Key Innovation - Magnetic Tape: The first coated tape method of audio recording was developed in 1927 by Fritz Pfleumer in Germany [timeline for magnetic recording technology]. The commercial development rights to Pfleumer's ideas were then acquired by AEG in 1932, and with assistance in plastic film technology from BASF a modern format reel-to-reel tape recorder [picture] was ready for demonstration at the Berlin Radio Fair in 1935. This breakthrough for the broadcasting and home entertainment industries was not lost on the emerging data processing industry, and magnetic tape readers and writers were included on the Eckert-Mauchly Computer Corporation's BINAC development in around 1949, and then on the 1951 UNIVAC, proving faster and more reliable than the earlier paper tape systems. << AUTHOR'S NOTE: Given that the overall purpose of this seven part paper is to study the penetration of computing concepts into cognitive theory, it is worth noting at this juncture that magnetic tape storage has little explanatory or metaphoric value in this respect. The essence of a tape as a tape is that it is dismountable, which biological memory most certainly is not, and the essence of the data stored upon it is that it is stored "serially", that is to say, there is no overriding logical sequence to the individual items stored nor is there any quick method of retrieving a particular record from mid-file - you just have to start searching at the beginning and keep looking until you chance upon it, which takes time. Biological memory, on the other hand, remains permanently online, and seems to rely on two considerably more flexible storage techniques, namely (a) "sequential" storage, where the individual records either arise naturally in, or have been sorted into, some clear logical sequence (time-stamped order, say), and (b) "random access" storage, where either an "index" or a "hashing algorithm" [details] allows a target record to be retrieved from mid-file. We therefore see little mileage in the tape metaphor, although our final evaluation will not be available until 2004. >>

�

1.1 The MIT Systems

Two of the very biggest innovations - ferrite core memory and the concept of real-time computing for command and control - came from MIT's Whirlwind team. Whirlwind's success for the US Navy in the 1940s [reminder], plus deteriorating relations with the then Soviet Union, prompted the USAF to commission an experimental air early-warning system from them. Whirlwind showed that such a complex linking of radar and computing systems was feasible on 20th April 1951, when it simultaneously tracked three separate aircraft from radar and computed their interception parameters (Waldrop, 2001) [the hard part, apparently, lay in getting the interceptors round behind their approaching targets]. This demonstration resulted in massive government funding for air defence technology, and the result (although it was not fully operationally deployed until 1963) was SAGE (= Semi-Automatic Ground Environment), "a continent-spanning system of 23 Direction Centres [picture], each housing up to 50 human operators, plus two redundant real-time computers capable of tracking up to 400 airplanes at once" (ibid., p75). The machines themselves were massive Whirlwind clones, code-numbered AN/FSQ-7, and each contained more than 50,000 valves and ran "the largest computer program written up to that time" (source). [For a fuller introduction to US air defence technology, see Boyne (1999/2003 online).]

�

BIOGRAPHICAL ASIDE - J.C.R. LICKLIDER: J.C.R. ("Lick") Licklider (1915-1990) was a psychologist-in-computing who made his name on the SAGE project. He had previously been with the Harvard Psychoacoustics Laboratory, where he had worked on measuring and improving the intelligibility of the voice communication systems available to bomber crews. He was thus one of the founding fathers of the signal-to-noise-ratio approach to perception which burst upon the psychological scene in the 1950s. He joined MIT in 1950 to work on the human factors aspects of multiple mainframe computing, and soon became convinced that it was only a matter of time and good design before properly trained operators would be able more or less to hold a conversation with their CPU. This was the birth of the "on-line transaction processing" type of modern computing, as already profiled [reminder]. In 1960, Licklider summarised these ideas in what is now seen as a classic paper, entitled "Man-Computer Symbiosis" (Licklider, 1960/2003 online), in which he wrote "..... the resulting [man-computer] partnership will think as no brain has thought and process data in a way not approached by the information-handling machines we know today". In 1962, Licklider moved to the Pentagon's Advanced Research Projects Agency (ARPA), to continue his work on distributed command and control systems, where he put forward detailed proposals to raise the connectivity of the various systems then in use, so that they could exchange data more easily. He left ARPA in 1963, leaving these concepts to be fleshed out and made to work by his successor, Larry Roberts, under the name ARPANET [more on which in Part 5]. Licklider also contributed towards MIT's �1963 "Man and Computer" (or "MAC") project [more on which in Part 5], which eventually evolved into the world's first "online community" (Waldrop, 2001), and wrote a number of influential papers profiling the sort of knowledge interrogation software which Internet search engines have since made commonplace.

�

1.2 The IBM Systems

As we have already seen in Part 1, Part 2, and Part 3, IBM had been the data processing corporation half a century before binary electronics was invented. By 1945, its punched card business had corporate customers worldwide, with storerooms full of data and processing suites full of card sorters and tabulator-adders. Yet these punched card archives largely defied complex data analysis, because the tabulator-adders were basically little more than large desk calculators fitted with card readers - they were like fish out of water when given multiplications and divisions to carry out. IBM's strategy after the war was therefore to see to its customers first and to do pure R & D second. This, after all, had been the strategy which had made them number one in the first place, and their CEO, Thomas J. Watson, saw no reason why it should not keep them there. So they took the new-fangled digital circuitry and installed it in their tabulator-adders, thus turning them into tabulator-multipliers such as the IBM 603 (1946) and IBM 604 (1948).

�

As for the pure R & D, IBM had been there with its chequebook for the 1937-1943 Harvard Mark I development [reminder], and had pushed through the 1947 SSEC [reminder] with no little urgency once Watson had decided that digital computing skills were too strategically important not to have in house. This was an inspired decision, and the SSEC experience immediately became the basis for the 1952 IBM 701 "Defence Calculator". This was a typical first generation computer, initially designed for the US defence market, and around 25 times faster than the SSEC it replaced. What made this machine historically significant is the fact that it also span off to become IBM's first top-of-the-range production machine for the civilian market, selling to the US Weather Bureau, aerospace companies, and the like. The project was directed by Nathaniel Rochester and Jerrier Haddad [team bios], and when the machine was formally announced on 29th April 1952 it came with enough CRT Main Memory for 2048 36-bit words, plus drum and tape backing store. Nineteen machines had been sold by the time it was replaced by the IBM 704 in 1955. The 704 was essentially a 701 upgraded to take the new and far more cost-effective ferrite core memory, and 123 of these were sold by the time it was replaced in turn by the IBM 7090 in 1960.

�

The 701 was a large system, both physically and functionally, and with a price tag to match. However, thanks to the vision of one of its then junior engineers, Cuthbert C. Hurd (1911-1996), IBM management were persuaded to include a scaled down version - the IBM 650 - in their sales portfolio. This was a smaller mass-market machine, lacking the expensive CRT RAM, but deliberately intended to be compatible with their tabulating machine range. The 650s retailed at between $200,000 and $400,000 according to the precise configuration, and around 1800 of them were sold (Gray, 1999/2003 online).

�

The IBM 650 was also honoured to be the first production machine to offer a random (or "direct") access disk memory option. To understand the true significance of this particular innovation, we must remind ourselves that our previous mentions of random access have related to main memory (where the words had even stamped the initials "RA" into the acronym RAM). When it came to backing store, however, the situation remained that files of any size would have to be stored as a succession of contiguous records - perhaps key-sorted, perhaps not - on magnetic tape [see earlier inset]; main memory could not (and, fifty years later, still cannot) hold more than a fraction of the total data available to the system. And the problem with magnetic tape, of course, was that as often as not several thousand feet of very delicate data and several minutes of waiting would be between you and the record you were after.

�

Now we have already seen how magnetic drum storage had been successfully implemented on the Harvard Mark 3 and the Manchester Mark I systems, and designers had already recognised that a spinning disk would have advantages over a rolling drum, if only the problems of size, cost, capacity, and reliability could be overcome .....

�

Key Innovation - Magnetic Disk: Inspired by the shape and ease of manufacture of the gramophone record, the magnetic computer disk was a platter (or stack of platters) of non-ferrous material, thinly coated with a magnetic oxide emulsion and accessed by a read-write head (or heads) mounted on the radius line. When installed as a "disk drive" computer peripheral, this technology allowed data to be stored in concentric circular "tracks", one beneath each position of the head(s). If you now took your giant tape file and wrote the records n-at-a-time onto as many tracks as it took, you could - providing you knew (or could in some way compute) the platter-track address - get hold of any one of them again in only a few tens of milliseconds, even if it were situated right at the end of the file.

�

The search for workable disk technology began in earnest at IBM in 1952, under Arthur J. Critchlow [IBM fact sheet and pictures] and the story has recently been re-told by Rostky (1998/2004 online). IBM announced the new technology on May 1955, and named it RAMAC. Each 50-platter drive stored what would today fit onto three standard floppy disks. It remains only to note that the direct access facility revolutionised both operating system design and application programming. To systems programmers direct access technology offered the sort of "paging" which would soon be used to deliver "virtual memory" systems [see Part 5; Section 1.2], and to applications programmers it offered not just disk-based files permanently online, but also "indexed sequential" organisation thereof capable of converting record key into a disk address. It also, in 1961, assisted at the birth of the modern database [see separate story].

�

1.3 The Remington/Sperry Systems

IBM did not have things entirely their own way, however, for they were facing increasing commercial competition in the early 1950s from the Remington Corporation [reminder]. In February 1950 and December 1951, respectively, Remington acquired the controlling interests in the Eckert-Mauchly Computer Corporation (EMCC), and Electronic Research Associates (ERA). EMCC, it will be recalled, was the company which the Johns Eckert and Mauchly had set up in 1946-7 to build BINAC and UNIVAC [reminder], and ERA was the company set up in 1946 by Howard T. Engstrom (1902-1962) and William C. Norris (1911-) to build the US Navy's 1947 "Task 13" machine (the machine which evolved into the US National Security Agency's October 1950 Atlas 1101 computer) [reminder]. And despite the fact that Remington's own computing department, the EMCC people, and the ERA people were geographically separate units, and retained a "high degree of autonomy, indeed rivalry" (Gray, 2002/2003 online), Remington had a minor star in EMCC's UNIVAC 1, which they duly capitalised upon by rebranding a demilitarised variant of the Atlas 1101 [picture] as the UNIVAC 1101, complete with a parallel organised arithmetic unit. They also won a second National Security Agency contract for the Atlas II (1950-1953), an Air Force contract for the (three-off) UNIVAC 1102 (1952-1956), and got government permission to market a civilian version of the Atlas II, to be called UNIVAC 1103 (Gray, 2002/2003 online). [Harry Wise was on that team, and has some technical recollections and photographs online - click here.]

�

This was all cutting edge stuff, of course, and it was projects like these which attracted the likes of Seymour Cray (1925-1996) into computing. Cray joined ERA in 1951, and impressed management so much that in 1953 they appointed him Project Engineer for the 1103 (Breckenridge, 1996/2003 online).� He rewarded their faith in him by turning the 1103A into one of the most innovative machines of its day. One development in particular - a "program interrupt" facility - was developed on an 1103A at the Lewis Research Centre, Cleveland (Gray, 2002/2003 online), an outstation of the US National Advisory Committee on Aeronautics (NACA) [more background].

�

Key Innovation - "Program Interrupts" (First Mention): According to Gray (2002/2003 online), the 1103A interrupt facility "permitted the 1103A to interrupt the processing of one program to handle another". This might be useful, for example, if an urgent job arose part-way through the execution of a much longer one, but where too much processing had already been done just to abandon it and start again later. To do this switchover safely, requires taking careful note of the position of the executing program, and then being able to reinstate all the registers and Main Memory settings to their pre-interrupt state afterwards, and this in turn requires sophisticated systems software and additional memory resources. We return to this topic in Part 5. << AUTHOR'S NOTE: The idea of one program interrupting another is quite popular in modern theories of high level motor control, especially those of the Norman-Shallice type. For specific examples, click here or here. The role of interrupts, however, is nevertheless poorly dealt with. Our full evaluation will not be available until 2004. >>

�

Further enhancements to the 1103A followed in the form of the 1958 1103AF, the first production machine to provide a reliable floating point arithmetic facility (Gray, 2002/2003 online) .....

�

Basic Terminology - Decimal Fraction: A decimal fraction is the sort of number we see all around us every day, that is to say, numbers such as 127.30 or 3.142. Each such number consists of two theoretically infinite strings of digits, either side of a "decimal point". The digits to the left of the decimal point are known as the "integer" part of the fraction, and the digits to the right are known as the "decimal". Taken together, these three elements are sometimes referred to as a "mantissa".

�

Key Innovation - Floating Point Arithmetic: Ever since the days of Schickard and Pascal [reminder], calculating machinery manufacturers have had to trade off complexity and cost against accuracy. If you decided to offer your customers ten-digit accuracy, for example, then someone wanting to multiply two such numbers together would suddenly need to display up to 20 digits, yet it would be wasteful to provide a register that big if it were only going to be used once in a blue moon. This is why Babbage's 1822 Difference Engine went for 31 digits and turned out to be an engineering nightmare, whilst the 1642 Pascaline made do with eight [reminder] and was well received. An added problem was that the number to be displayed might be all integer, all decimal, or part and part, so calculator designers also had to keep track of where the decimal point should go, which they did by "fixing" it, and by leaving it to their customers to do "fixed point arithmetic". Things were no less troublesome for the computer designers of the 1940s, especially those working in binary (because base 2 numbers are roughly four times as long as their base 10 counterparts), and the eventual solution was to convert all numbers into what is called "scientific notation", a numbering convention where only one significant digit is allowed in the integer part of the mantissa, and in which the true scale of the resulting number is then obtained by multiplying it by an associated power term. Each power term consists of a "base" - typically 2 in binary or 10 in base 10 - and an "exponent", the power to which the base must be raised. Thus .....

�

0.147 is the same thing as 1.47 multiplied by 0.1, and can be expressed as 1.47 x 10^-1

147 is the same thing as 1.47 multiplied by 100, and can be expressed as 1.47 x 10²

14700 is the same thing as 1.47 multiplied by 10000, and can be expressed as 1.47 x 10⁴

�

..... and so on. The decimal point, in other words, is now free to "float" from wherever it started off until it arrives at its standardised new position, leaving the power term to remember where it had been. The "precision" of a given system is the number of significant digits catered for in the mantissa. Thus the number 14741 in a precision-3 system (as shown above) loses the fourth and fifth significant digits and will be treated as 14700, and this means that we risk introducing "rounding error" into our arithmetic if we throw lots of significant digits at a machine which has not been set up with the precision necessary to handle that number of digits safely. However, the advantage of scientific notation is that it greatly simplifies the task of computerising the multiplication process, because your two multiplicands will now always lie between 1 and 9 (and can therefore easily be fixed point multiplied), and the power terms can be multiplied by adding the two exponents. Thus a precision-2 multiplication of 0.0201 by 35192 will proceed as follows .....

�

(1) Convert each multiplicand to scientific notation, thus .....

�

(2.0 x 10^-2) x (3.5 x 10⁴)

�

(2) Do the multiplication as described above, giving .....

�

7.0 x 10^-2 x 10⁴

�

(3) Calculate the new exponent as described above, giving .....

�

7.0 x 10²

�

(4) Convert the answer back into everyday form, giving .....

�

700

�

As it happens, the arithmetically correct answer is 707.3592, so we are about 1% out, the difference arising from the aforementioned rounding error. It would therefore be wise to increase our precision, and a precision-4 multiplication of the same two numbers duly gives a better approximation, namely 707.3 [you actually need a precision-7 multiplication to remove the error altogether]. Unfortunately, rounding error is endemic in floating point arithmetic, so for scientific applications the precision is nowadays set high at an IEEE-recommended 32 bits for "single precision" working, or 64 bits for "double precision" working [details]; and even then things can go wrong, as when an unexpected rounding error of this sort contributed to a failure in a Patriot missile launch during the First Gulf War, resulting in a Scud strike on a US barracks, killing 28 servicemen [details]. As Goldberg (1991/2003 online) puts it, "squeezing infinitely many real numbers into a finite number of bits requires an approximate representation", thus presenting the computing industry with the intriguing paradox that the faster you can do your calculations the more you need to worry whether one of these inherent inaccuracies has taken place. [For a fuller introduction to the intricacies of floating point arithmetic, complete with worked examples, click here.] << AUTHOR'S NOTE: The floating point concept - perhaps surprisingly - is not totally incompatible (mutatis mutandis) with what is known about modular mathematical cognition. As we have explained elsewhere (Smith, 1998), mathematical cognition consists of at least two basic "left hemisphere" processes and at least one "right hemisphere" process, all of which seem capable of simultaneously working on a given mathematical task. The left hemisphere processes are (a) to establish the conceptual meaning of the numerals and arithmetical symbols used, and (b) to activate and put into effect the rule sequence by which the solution might be derived. The right hemisphere process is to predict the result of the overall calculation by roughly estimating what the answer should be. In addition, the ability to use the "number line" may provide both conceptual (left) and visuo-spatial (right) support. Our full evaluation will not be available until 2004, but, in the meantime, interested readers may find no little fascination in the following neuropsychological case study data - click here. >>

�

In 1955, Remington merged with Sperry Gyroscopes to form the Sperry-Rand Corporation, and in 1956 the 1103 was supplemented by the markedly smaller UNIVAC "File Computer". This machine got its name from another important innovation, namely the long-term storage of large commercial data files on magnetic drum, rather than on punched card - thus mounting a direct assault on IBM's hitherto captive market. Not only could large bodies of commercial data now be rolled forward on a daily, weekly, monthly, or yearly basis, but it would henceforth no longer be necessary to trolley the files in and out of the machine each time they were needed.

�

Key Innovation - Periodic Batch Processing: Accountants and bankers have had to "carry forward" balances from the end of one accounting period to the beginning of the next ever since money was first invented, and the basic principles of book-keeping within such an accounting period are laid out in Luca Pacioli's "Summa de Arithmetica" (1493). �[William Goetzmann, of the Yale School of Management, places the first recognisable bank in Uruk in Mesopotamia (modern Iraq) around 5,000 years ago; click for details; picture ; for a delightful and totally fascinating history of the profession of accountancy, click here.] The calculations, however, were done by hand until the advent of mechanical calculators towards the end of the 19th century [reminder], and - if you were lucky enough to be able to afford it - were then semi-automated by the Hollerith style punched card industry of the early 20th century. Now the point about punched card processing is that data flowed from storage point to storage point via machine after machine and through tray after tray, being merged, sorted, updated, copied, collated, resorted, etc. Finance Departments suddenly had to be totally precise (a) in how they allocated their records - the cards - into files, and (b) how the files then metamorphosed as they passed through the various processing stages. Fortunately, although the nature of the data varied widely according to the nature of the business involved, there were some clear basic patterns. By far the greatest volume of data was "transaction data", that is to say, individual records of individual happenings during the period in question - individual sales, individual deliveries, individual money transfers, etc. However, this type of data only really meant anything when seen in the context (a) of the situation at the end of the preceding period (as set out in the opening balances contained in some sort of permanently maintained "master file"), and (b) of supporting detail such as rate books ("reference data") and customer details (as set out in some sort of "customer file"). Thus the sequence of events in a typical 1920s stock system might be to open up a master file of opening stock levels (containing one record card per item per location), and then to "tally forward" each stock level by the receipts, issues, and adjustments of that item contained in one or more "transaction files". At period end, this tally forward would generate the theoretical carried forward stock levels, which could then be audited against a physical stockcheck, if necessary. The experience of the Lyons Company's office systems guru, John Simmons, is probably indicative. He had been appointed in 1923, remember, in order to bring scientific rigour into the design and operation of Lyons' corporate processes [to streamline, first and foremost; to automate if that helped streamline; and to computerise if that helped automate], and their 1949 LEO project [reminder] succeeded in automating the batch processing routine for payroll applications in December 1953 [more on this in Section 2.4]. << AUTHOR'S NOTE: The standard tool for the specification of these large batch processes was the dataflow diagram (DFD) [background and example]. DFDs are now the diagram of choice for tracking the movement of information (a) through an extended system, and (b) through the processing stages carried out at any one node within that system. As for the use of flow diagrams to help explain cognition, they are already standard practice (albeit they are not always very well done technically). We have written at length on this subject elsewhere, and the subject can safely be approached from any of three directions, for the material constantly cross-refers. To begin with a specific example of a modular cognitive system, click here, or to begin with an introductory commentary, click here, or to begin with some back-to-basics practical diagramming, click here. >>

�

Lyons' and Sperry's pioneering work led to a growing confidence that large corporate files could be safely committed to a medium which was inherently unreadable by humans, and this was commercially highly significant because it enabled the new electronic systems to threaten the data tabulating industry in its own back yard; to offer an upgrade pathway for the literally thousands of tons of legacy data still held on punched cards. The design spotlight therefore fell on such factors as storage capacity, processing speed, and system reliability, and it soon became standard design practice to assess whether the file handling and the number crunching aspects of a given process presented broadly balanced demands. If they did, then all was well, but if either was slow compared to the other, then you had to suspect a bottleneck of some sort .....

�

Key Innovation - I/O vs CPU Bottlenecks: New terminology soon emerged to describe the various problem scenarios. If a process struggled when reading data in or when writing data out it was referred to as "peripheral bound", but if it struggled during the number-crunching phase it was referred to as "processor bound". When the present author joined British Telecom in 1980, he learned of "the Great ISOCC Tea Trolley Disaster" [English humour]. ISOCC was the BT computer system then responsible for charging for operator-connected calls. For a time, however, "telephone tickets" [picture] were being produced by the nation's telephone operators faster than the billing centres could process them; specifically, it took about 25 hours to process every day's input slips, so the processing backlog simply got longer by the day. We can no longer recall with certainty whether the problem lay in getting the data punched into the machine (in which case the process would have been peripheral bound), or in carrying out the necessary calculations once it had been (in which case it would have been processor bound), but it was probably the former.�<< AUTHOR'S NOTE: Neuroscience does not know enough about biological processing architectures to judge whether the unit of binary decision making - that is to say, the biological equivalent of a logic gate - is a single neuron, a small neuron cluster, a large neuron cluster, an entire gyrus or ganglion, or an entire cortical layer. It is conceivable, indeed, that structures traditionally ignored by neuropsychologists, specifically the brain's "white" as opposed to "grey" matter, or even the glial supporting cells, may also be able to act in some way to modulate what is going on in their electrically noisier neighbours. We raise this as an issue, because until neuroscience can identify the logic gates it will remain unable reliably to identify the larger logic or control units either, so it is difficult to agree on which brain areas count as peripherals and which as central systems, and therefore which is getting overloaded. Interested readers may obtain some idea of current thinking in this area from the following website, click here. We extend the argument when discussing serial and parallel computing in Section 2.2. >>

�

Hollingdale and Tootill (1965) explain the problem, and its conventional solution, this way .....

�

"Before 1958 most computers were engaged on scientific or technical calculations of one kind or another; nowadays most of them are employed on clerical tasks - on payroll or stock control calculations, for example. In this kind of work the essential role of the computer is to manipulate large quantities of digitally coded material - usually referred to as data - in a prescribed manner [.....] The amount of computation that has to be done on each unit of data is usually quite small [..... //] The organisation of the rapid input and output of streams of coded data is, in fact, one of the major problems of [commercial computing]. [.....] The basic idea is to carry out all the slower input and output processes away from the main computer, whose precious time must not be squandered on sluggish menial tasks. These are delegated to a subsidiary computer, the essential function of which is to transcribe information from one physical medium to another ....." (p239-240.)

�

The upshot of all these problems was that Lyons and Sperry perfected some historically important methods of coping more effectively with peripheral bound processes by strategically placing additional memory resources about the system .....

�

Key Innovation - "Buffers" and "Buffering": "Buffers" in general are simply areas of temporary storage somewhere in size between a register and a decent slice of Main Memory, and they are important because they allow "buffering", that is to say, "the temporary holding of information in transit" (Purser, 1987, p312). Like any other form of digital storage, a buffer consists ultimately of a contiguous series of flip-flops, and the underlying technology may vary from machine to machine. To understand how buffering works, we need to remind ourselves (a) that Eckert-von Neumann GPCs [reminder] habitually process their input and output files through reserved record areas in Main Memory, (b) that file reading and writing are comparatively slow processes, and (c) that Lyons' and Sperry's customers were trying to funnel enormous volumes of data through card or magnetic tape readers. Bit String Buffering: The Lyons and Sperry designers therefore added what was effectively a small free-standing register between the relatively fast data bus and the relatively slow peripheral device controllers. As data arrived in the form of a slow stream of bits from a peripheral device, it was loaded into an "input buffer", and only when that buffer was full was the accumulated stream released - either at a higher serial rate, or in parallel - onto the bus to complete its journey to its destination in CPU or Main Memory. This short delay left the CPU free to be getting on with something else. A similarly arranged "output buffer" could buffer data en route from the output record area to an output device. Pinkerton (1990/2003 online) profiles the buffering system on the LEO as follows: "The buffers in each of the input channels were all valve circuits and mercury delay lines. They had little short tubes which assembled individual numbers and put them into position, one after the other until they filled it up. The same thing on the output side; some of the buffers were doubled on the input though not on the output. The logic allowed four input channels and four output channels ....."�Main Memory Buffering: It is also possible to use Main Memory for buffering. This can be done by doubling (etc.) the number of Main Memory record blocks allocated to each file, and by then routinely loading these one record ahead of the ongoing processing. This sort of "double buffering" introduced a degree of record-level overlap into the system, such that the input delay for record <n+1> coincided with the processing time for record <n>, which, in turn, coincided with the output delay for record <n-1>. For very fast processes, "treble buffering" (etc.) could be resorted to, thus creating even more overlap.�It follows that a given input character was likely to have been buffered twice on its way to the CPU, once at the device stage, and then again at the Main Memory stage. Instruction Buffering: We deal with another type of buffer - the "instruction buffer" - in Section 2.3, under the heading Pipelining. "Store and Forward" Buffering: This is a form of buffering which occurs (in addition to all the others) at the subordinate nodes in a communications network. It is needed whenever three or more computers have been interconnected, and it allows transmissions to be stored temporarily at node #2 en route for node #3, should the line to node #3 be down or otherwise engaged, or should the CPU at node #3 be too busy to accept the input immediately. Modern telecommunications - including the technology currently bringing you this page - devotes much of its attention to such problems, and network designers need to carry out a complex traffic queuing analysis before deciding how much buffering store to provide at each intermediate node. << AUTHOR'S NOTE: The concept of output buffering is heavily used in modern theories of motor behaviour - for specific examples, click here, or here, or here. The concept is less well handled in theories of perception, although the phenomena (a) of perceptual memory [see Sperling (1960)] and (b) of saccade-based text processing [see Gough (1972)] are both prime candidates for it. It is also poorly handled in matters of human communications, where the "store and forward" version of buffering is not catered for in any of the common forms of the "sociogram". This, as we have repeatedly argued elsewhere [eg. Smith (1997; Chapter 1)], seriously erodes the explanatory value of most modern cognitive models [for our most recent offering on this topic, see our e-paper on "peer-to-peer" telecommunication]. >>

�

2 - Computing 1951-1958 - First Generation Hardware (UK)

The 1945 demobilisation took all but a select few of the British wartime engineers from their top secret work and returned them - tightly sworn to silence by the Official Secrets Act - to peacetime employment. We begin this phase of our story with one of these demobilised boffins, Andrew D. Booth, one of the pioneers of magnetic drum memory .....

�

2.1 The Birkbeck and "British Hollerith" Machines

Andrew D. Booth (1918-) was a Birkbeck College physicist who, as an employee of the British Rubber Producers Research Association, had been involved in X-ray crystallography during the war .....

�

ASIDE: The British Rubber Producers Research Association (BRPRA) was founded in 1938 to promote formal scientific knowledge of said product. It set up research centres in both Malaya and Welwyn Garden City, Hertfordshire [pictures], and one of its first researchers was Cardiff-born George Alan Jeffrey (1915-2000), later Professor of Crystallography at the University of Pittsburgh. Now the point about rubber is that it is a highly strategic commodity, perfectly capable of winning or losing wars. Without it, for example, you have no tyres for your vehicles or aeroplanes, no engine mountings, no oil seals, no hoses, no rubber dinghies, etc. Strategically, therefore, you either have to own the hot places in the world where the product can be grown naturally, or you have to have the raw materials and industrial base to manufacture it synthetically. Either way, you need a first class team of physicists and chemists trying to make the stuff longer lasting and constantly analysing specimens cut from wrecked enemy equipment to see if they had managed to do the job better than you had (much of Jeffrey's war was spent X-raying slices of rubber cut from wrecked German aircraft). It was this strategic war work which the BRPRA was there to organise, and, humble though the job might sound, the rubber boffins probably contributed every bit as much per capita to the war effort as did the codebreakers at Bletchley Park. It was Jeffrey who recruited Booth, and it was the fact that X-ray crystallography creates raw data like there is no tomorrow that led Booth to develop automatic calculating machinery as a laboratory aid to data analysis in much the same way that Wynn-Williams had scratched together his thyratron circuits at Cambridge in the early 1930s [reminder].

�

Booth moved to Birkbeck College, University of London, after the war, and in 1946 his automatic calculating work brought him to the attention of the US National Defence Research Committee's chief "talent scout", Warren Weaver [bio follows in Section 4.1]. Weaver was intrigued by what he saw, and put Booth forward for a study scholarship under John von Neumann at Princeton's Institute of Advanced Studies [a cynic might ask in retrospect who was studying whom]. Accompanied by his research assistant Katherine H.V. Britten (whom he subsequently married), Booth was at Princeton from March to September 1947, and upon his return to Britain proceeded to build a small relay computer named the Automatic Relay Computer (ARC) (Lavington 1980/2003 online). He demonstrated this machine to Weaver on 23rd May 1948 at BRPRA's Welwyn Garden City centre (Hutchins, 1986/2003 online), and we shall be finding out what Weaver did next in Section 4.1. But what made the ARC more than just a British version of, say, the Bell Mark II relay computer [reminder] was the fact that Booth had cleverly linked it to a simple magnetic drum memory .....�

�

Key Innovation - Magnetic Drum Storage: The advantage of magnetic drum storage over tape lies in the fact that the storage surface rotates continuously against a linear array of read-write heads [picture]. This means that your data is re-available to you once per revolution, unlike tape-borne data, where there is a lengthy re-wind time between passes. The sequence of magnetically coded pulses recorded by any one head is known as a "track", and each stored bit offers itself back to the head for reading and/or overwriting every few milliseconds [this rotational delay being known as its "latency"]. The system has quite a history, in fact, for the idea of "rolling up" your data storage in order to save space is as old as the scrolls of ancient times, and the idea of winding data around a cylinder, so that it spirals gradually from one end to the other, is clearly seen in both the Greek cryptographical system known as the scytale [reminder] and Edison's famous cylinder phonograph [picture]. It is therefore not surprising that when, in 1898, the Danish inventor Valdemar Poulsen was looking for somewhere to keep his magnetised wire recordings, he chose a drum [picture]; nor that the computer pioneer John Atanasoff toyed with the drum concept in the late 1930s on the Atanasoff-Berry Computer [reminder], albeit he ended up using surface capacitors as his storage substrate rather than a magnetic coating [picture] (Gray, 1999/2003 online). What is not so clear is who first applied Pfleumer's ferrite emulsion as a magnetic lacquer coating to a drum, and the first clear suggestion to this effect seems to have been in an MIT master's thesis by one Perry Crawford.

�

BIOGRAPHICAL ASIDE [INCOMPLETE] - PERRY O. CRAWFORD (? - ?): According to Jay Forrester, the Whirlwind Project Director, it was Crawford who first called his attention to the possibility of digital computation (Forrester, 1988/2003 online), this having been the subject of his MIT master's thesis during the period 1939? to 1942?. Mindell (2002) mentions Crawford as having succeeding Claude Shannon as postgraduate student at MIT, sometime around 1940, and it was in the period 1940 to 1942 that Crawford developed his ideas on the use of digital methods to fire control problems. During 1942 to 1945 Crawford served (as an attached civilian) with the navy's Special Devices Centre, later at Sands Point, Long Island, building flight simulators and training devices and coming up with the name "Whirlwind" into the bargain. In October 1945 he went to work for the Special Services Division of the Office of Naval Research. Crawford left the navy in 1952, and moved to IBM, where he became involved in the SABRE project [see Part 5]. He was still around to be interviewed in 1995 (by Mindell), but seems to have published little.

�

Booth experimented with the drum concept in his Birkbeck College laboratory in 1946-1947, and passed on the details to the Manchester University team who added it to their Baby [reminder] (Booth, 1995/2003 online). On balance, however, it is probably fairer to credit the first large scale investment in drum technology neither to MIT nor Birkbeck, but to ERA's John Coombs, as part of the US Navy's Project Goldberg, also in around 1946. ERA used a 5" drum revolving at 3000 rpm, and set their packing density at 230 bits per inch [source]. The literature also mentions an 8" drum developed in 1948 by the University of California at Berkeley (Spice, 2001/2003 online) and recording at 800 bits per inch, and other units on the Harvard Mark III and the SEAC [source].�<< AUTHOR'S NOTE: The notion of a constantly recycling memory is superficially similar (a) to the psychological concept of the "rehearsal loop", and (b) to the physiological concept of the self-sustaining neural "reverberatory circuit". Rehearsal has been suggested by many authors as a mechanism whereby a small number of auditory items can be maintained in memory by constant covert repetition. Our full evaluation of the literature here will not be available until 2004, but to see rehearsal loops in a cognitive model click here. >>

�

After making his name with the ARC, Booth moved up to thermionic valves, trying out his circuitry in a 1949 prototype machine called the Simple Electronic Computer (SEC). When he was happy that he knew what he was doing, he followed this up with a number of�larger, but still drum-equipped, machines in his 415-valved APE*C Series, where the APEC stands for "All Purpose Electronic Computer" and the asterisk indicates the particular commercial sponsor at the time. Thus the APERC [picture] was delivered to the British Rayon Research Association, the APEHC was delivered to the British Tabulating Machine Company ("H" because BTM were the holders of the Hollerith licence in Britain), the APENC was delivered to a Norwegian sponsor, and the APEXC was his own Birkbeck machine. The APEHC was BTM's first excursion into electronic computation, and they quickly reworked it in house as the HEC [picture]. They then - goaded no doubt by IBM's success with its electronic multipliers [see Section 1.2] - used the HEC experience to build punched card processors of their own, such as the BTM 541 (1953) and BTM 555 (1957) [pictures]. The HEC was also the inspiration for the BTM 1200 (1954) and BTM 1201 (1957) computers, and thus indirectly for the ICT 1300-series and ICL 1900-series mainframes which followed them. [Francis Townsend was part of the team responsible, and has some technical recollections and photographs online - click here.]

�

2.2 The Last of the Boffins

So what of the boffins who had been kept on after the war in the secretive government and military research stations we heard so much about in Part 3. Well basically we spoke of four such sites - Bletchley Park, the National Physical Laboratory (NPL), Dollis Hill, and the Research Establishment at Malvern, Worcestershire, so there are only four stories to tell, and we shall take them in order of mention.

�

After the war, the GCCS at Bletchley Park continued to develop its Colossus machines, sold an unspecified number to friendly foreign governments [which is one of the reasons their existence was not acknowledged for such a long time], and remained active in the security business for many years. In the end, however, the unit was relocated and the site itself has recently become a museum [to see what Hut 4 is doing nowadays (31st May 2003), click here]. But GCCS made no further computers [or, if it did, they are still classified], and so it now moves outside the remit of this paper.

�

We left the NPL in 1949, struggling to get their cool but decidedly complicated Pilot ACE up and running [reminder]. In 1948, Turing had been transferred to Max Newman's laboratory at Manchester, and the Pilot ACE continued in development for two more years under James H. Wilkinson and E.A. ("Ted") Newman. It executed its first program on 10th May 1950 (making the NPL, they claim, the fourth group in the world to deliver a stored program electronic digital computer), and was publicly demonstrated in December that same year. The NPL then transferred the production rights in Pilot ACE to the English Electric Company [company history], the then owners of the Marconi interests, who tidied it up a bit before putting it to market as the Digital Electronic Universal Computing Engine (DEUCE), eventually delivering more than 30 machines at around �50,000 apiece [source]. A full (84-page!) technical specification of the DEUCE is available online. The NPL's (Full) ACE was ready in 1958, still with delay-line storage, whereupon it was decided that there was no longer any role for government owned mainframe development. Full ACE thus marked the end of the NPL's involvement in mainstream computer development, so it, too, now moves outside the remit of this paper.

�

ASIDE: Rumour - probably not totally without foundation - has it that the Royal Aircraft Establishment actually bought two DEUCEs, so that they could run their really important programs on both of them, only trusting the results if both machines came up with the same answer (Kershaw, 1994/2003 online)! For more on Pilot ACE, see Newman (1994/2003 online), and for DEUCE, see Fisher (2003 online).

�

As for the Post Office boffins at Dollis Hill - that is to say, Tommy Flowers, A.W.M. Coombs, and their respective teams [reminder] - their fate is easy to relate because as telecommunications engineers in the first place they simply reverted to their former existences. Nevertheless there were occasional opportunities for off-site adventure, as when Coombs was called in to assist at .....

�

..... the War Department's own high-tech installation at Malvern, home, as we mentioned briefly in Part 3, of two of Britain's early defence computers. The first of these was a derivative of Pilot ACE called MOSAIC, sponsored by the Ministry of Supply for missile tracking applications. This was a 6000-valve, 2000-transistor, serial processing machine built between 1947 and 1954 by the Post Office's A.W.M. Coombs and team (Lavington, 1980/2003 online), and its Main Memory comprised 1024 40-bit words in mercury delay lines ("nearly a ton" of the stuff, according to Lavington). The second British defence computer was a more technically adventurous parallel processing machine called TREAC. This was designed in 1947 by Albert M. Uttley, went operational in 1953, and used a 512-word Williams-Kilburn system for fast store. The Australian Computer Museum also remind us of the work of Trevor Pearcey (1919-1998), a TRE veteran who in 1946 emigrated to Australia, where he designed the 1949 Australian CSIRAC computer, since claimed as the fifth stored program computer to be completed.

�

Key Innovation - Serial vs Parallel Processing: To do things "in series" is to do them one after another. To do things "in parallel" is to do them simultaneously. Serial computers therefore execute their data moves and their arithmetical operations one after another, whilst parallel computers do them simultaneously. As originally conceived, parallel processing computers used one wire per bit when moving whole words of data at a time between their registers and their logic gates, whilst serial computers used one wire only, and moved one bit at a time. The parallel machines were accordingly much faster, but much more electrically complicated. EDSAC's Maurice Wilkes explained the differences this way .....

�

"The most obvious division of computing machines is between those which are serial in operation and those which are parallel [but] the subject is an endless one in that many different variants of each kind of machine are possible. I do not consider that the question of whether it is better to build a serial or a parallel machine will ever be finally answered; each type has its advantages, and the final decision will always be dependent on the designer's personal preferences, and to some extent on his terms of reference. Also, as time goes on, the balance of advantage and disadvantage will swing as improvements are made in this or that component, or as entirely new components become available. Certain fundamental comparisons between serial and parallel machines can, however, be drawn readily enough. [//] The arithmetic unit of a parallel machine tends to be much larger than the arithmetic unit of a serial machine, since a separate flip-flop or equivalent circuit element must be provided to accommodate each digit in the various registers. Moreover, since addition of the digits takes place simultaneously, or nearly so, a separate adder must be provided for each digit. The disparity [.....] increases as the fundamental number length is increased. [.....//] The control, on the other hand, is much simpler in a parallel machine. In the first place the necessity for the generation of a set of digit pulses does not arise, and secondly, the timing is much simpler, since a number can be moved [.....], or an addition performed, by the application of a single pulse to a set of gates ....." (Wilkes, 1956, p66)

�

What we are looking at, therefore, are alternative ways of organising a single CPU - serial for cheap but slow, or parallel for complicated and expensive but fast; it is just another design decision to be made in the early stages of a machine development project. << AUTHOR'S NOTE: The parallel processing concept used by TREAC and the US parallel machines [Whirlwind and the 1101] is NOT THE SAME as the "parallel distributed processing" talked about by the PDP school of cognitive scientists [see our e-paper on Connectionism]. A parallel wired CPU moves whole data words at a time WITHIN A SINGLE CPU, but PDP as an incarnation of modular cognition REQUIRES MORE THAN ONE CPU. Or to put it another way, the essence of "parallel" in a parallel CPU is that many bits are moved simultaneously under the control of a single Control Unit, whilst its essence in a PDP configuration is that more than one Control Unit is at work. Now when we look at the brain, there is certainly no doubt that data moves simultaneously within even the smallest aggregate of neurons, but as we do not yet know the unit of binary decision making it would be unsafe to classify this as parallel logic. There is also certainly no doubt that different brain areas are simultaneously active during different types of cognitive task, but until you know which modules count as Control Units [and that is a philosophical problem as much as an engineering one] you cannot be sure what these resources are each contributing, or, indeed, to what extent they are in fact co-operating. Moreover, because each component CPU can be serial or parallel within itself, you can theoretically have PDP in a network of simultaneously active serial CPUs, albeit this would be slower than PDP in a network of simultaneously active parallel machines. You could even have PDP in a part-serial, part parallel, processing network. Our full evaluation of the literature pertaining to this issue will not be available until 2004, but there is much to learn in the meantime from the literature on brain injury. See, for example, our timeline notes on the nineteenth century neurologists Dax and Lordat, or the debate between the localisers and the globalists [timeline]. Note especially that both the hierarchical block diagram of Kussmaul (1878) and the transcoding block diagram of Charcot (1883) both derive from this type of evidence, and therefore so, too, do modern derivatives thereof, namely Norman (1990) and Ellis and Young (1988) respectively. >>

�

2.3 The NRDC Machines

Another cluster of first generation computers evolved out of the tripartite collaboration of Manchester University, the Ferranti Company, and Elliott Brothers, and the coordinating force here was the Government's scientific funding office, the National Research and Development Corporation (NRDC) [not to be confused with the American NDRC mentioned in Section 2.1 above and seen again in Section 4.1]. This organisation was set up in 1949 by the then Trade Minister Harold Wilson (later Prime Minister between 1964 and 1970, and 1974 to 1976), under the direction of John Anthony Hardinge Giffard (1908-2000), Third Earl of Halsbury, in an entirely laudable attempt to stretch the British taxpayer's return on state-funded research. This meant co-ordinating the work of the remaining defence establishments with interested parties throughout industry and academia, and the newly born computing industry was high on the NRDC's list of strategic priorities.�

�

ASIDE: In a blistering appraisal of the whole NRDC experience, Hendry (1989) explains how most of the companies the government helped had gone out of business by the mid-1960s [and the survivors - as we shall be seeing in Part 5 - have either gone since, or are pretty sick]. For a number of reasons, some screamingly obvious but others quite subtle, the NRDC tended to "help to no advantage". It transpired that neither government gifts, nor advantageous development loans (nor, indeed, roving facilitators) were particularly effective instruments of public policy. "At no time," Hendry concludes, "did the aid given or offered to any firm make any perceptible difference to that firm's business strategy" (p168). In the ten years to 1959, the NRDC distributed just short of �1 million in cash, and underwrote loans for a further �1.25 million. This compares with US military patronage for the US computer industry during the same period of "over $1 billion" (p161). For his part, Lord Halsbury described the whole experience as rather like "pushing mules uphill" (Halsbury, 1991, p272), and had little positive to say about the quality of the decision making in the companies he had been trying to assist. The Ferranti company was founded by Sebastian Ziani de Ferranti (1864-1930) in 1889 in London, to manufacture electrical generating equipment. During the First World War they produced munitions, and in the 1920s moved into radio and consumer electricals. The radio experience then stood them in good stead come the 1930s, when the RAF needed re-equipping, and again when the radar contracts started to come through shortly afterwards.

�

So let us start with the collaboration between Manchester University - whose Mark 1 we have already spoken about in detail [reminder] - and the local Ferranti works at Moston (later West Gorton), Manchester. The strategy here had already been hammered out at an NRDC meeting on 14th December 1949, and involved Ferranti preparing the Manchester Mark I for market as the Ferranti Mark I, thus freeing up the academics to push ahead with further technical developments. Manchester University therefore began the decade by getting on with the Manchester Mark II, alias "MEG". This incorporated a number of relatively minor developments, but when it first operated in May 1954 was nevertheless 20 times faster than the Mark I, thanks primarily to having changed from serial to parallel logic (Lavington, 1978/2003 online). This, too, was eventually marketed by Ferranti, as the "Mercury", and sold fairly successfully in competition with the IBM 704, because despite being a lot slower it was also a lot cheaper (ibid.). The university then moved on to the Manchester Mark III, an experimental transistorised machine, which contributed greatly to the 1956 (six-off) Metropolitan Vickers MV950, which has been claimed as the first operational transistorised computer. [We shall be dealing with Manchester's later machines, the second generation MUSE and Atlas, in Part 5].

�

Other work, meanwhile, had been going on in London and the Home Counties, the industrial player here being Elliott Brothers .....

�

HISTORICAL ASIDE: Elliott Brothers were founded by William Elliott in 1795 to supply Royal Navy officers with precision technical instruments, and had then simply grown with the technology to provide many of the Royal Navy's fire control systems during the analog computing era [see Part 2], not to mention supplying some of the parts for Babbage's original Difference Engine (Clarke, 1975). After World War Two, they were keen not to lose this marketing foothold, and in 1946 established a digital computing research unit of their own at Borehamwood, Hertfordshire. They selected John F. Coales (1907-1999), who had just spent the entire war on Elliott's gunnery radars, to run this unit, and their first major project was to develop a state-of-the-art computerised naval anti-aircraft fire control system, codenamed MRS. The broad specification here was for "ship-borne radar [Elliott's specialism, remember] [to] lock on to a target at about eight miles and then track it and supply aiming information to a ship's guns" (Lavington, 2000, pp8-9). A large number of R & D staff were duly taken on, and the computer which resulted was the Elliott 152 [picture], with logic circuits designed by Charles E. Owen (details in Clarke, 1975). The 152 "executed several instruction streams in parallel from an electrostatic read-only memory" (Lavington, 2000, p11). Government contracts being what they are, however, minds soon changed, and MRS was cancelled in 1949, threatening to put over half of Elliott's researchers out of a job (Hendry, 1989) .....

�

Now it so happened that the collapse of the MRS project coincided roughly with the inaugural meeting of the NRDC, so it made good political sense for the government to try to keep Coales' team together. The NRDC therefore chose Elliott to turn its now idle 152 into a medium-sized computer to complement the larger Manchester machines, issuing the first of a number of contracts in September 1950. The result, in 1953, was the Elliott 401, the prototype of a series of five increasingly sophisticated "400-Series" machines. The 401 was designed by Andrew St. Johnston as a valve-based computer with delay line and magnetic disc memory, and ran its first program in March 1953. It had a front-loading modular design [picture] for ease of maintenance, achieved high availability figures, and actually remained in use until 1965. [For more on the 401, see Lavington (2000). The machine itself is now preserved at the Science Museum, London.]

�

Key Innovation - Magnetic Disk Storage: One of the 401 innovations was the first magnetic disk. This was designed by Chris Phillips as a more compact and easier to engineer form of magnetic medium than the magnetic drum. However, the functional principles - track-based storage, access latency, etc - were the same as for the drum. << AUTHOR'S NOTE: Comments as for the magnetic drum [see Section 2.1]. >>

�

The 401 was followed in 1955 by the 402 and 403 production machines, the 403 being historically noteworthy because it fielded an early version of "pipelining" .....

�

Key Innovation - "Pipelining" and Related Concepts (First Mention): "Pipelining" is a technique "whereby multiple instructions are overlapped in execution" (Patterson and Hennessy, 1996, p125). We have already seen how the buffer concept allowed a temporary memory resource to be used to speed up I/O [see Section 1.3]. The Elliott 403 designers simply applied a similar concept to the instruction stream as well, arranging for the FETCH part of the basic FETCH-and-EXECUTE cycle [reminder] to be a step ahead of the EXECUTEs. This was accomplished by providing an "Instruction Buffer", to contain instruction <n + 1> in FETCH mode, while the Instruction Register [reminder] contained instruction <n> in EXECUTE mode. Needless to say, when instruction <n> had been successfully executed, an important decision needed to be made - had that instruction resulted in any form of program jump? If it had not, then the operands corresponding to the presumed next instruction, that is to say, instruction <n + 1> in the Instruction Buffer, could safely be used, and were duly copied to the Instruction Register in a ready-to-execute state. If, on the other hand, a program jump had been requested, then the presumption had been wrong, and the contents of the Instruction Buffer were unsafe, had to be discarded, and a new FETCH operation done to initiate the program branch. Since the safes were generally more frequent in practice than the unsafes, the technique delivered real savings in speed of processing. [We return to these concepts in Part 5, when dealing with the IBM 7030 and CDC 6600.] << AUTHOR'S NOTE: Neuroscience does not know enough about biological processing architectures to judge whether the brain has devised some sort of op code / operand system for itself, and accordingly to judge whether it does pipelining or not. We suspect that it does, allowed to by its own internal modularity and forced to by its very slow clock speed, but the full evaluation will not be available until 2004, and in the meantime interested readers may obtain some idea of current thinking in this area from the following website, click here. >>

�

The last of the Elliott 400-Series was the Elliott 405. This sold well both at home and internationally [possibly under the name NCR 304?], thanks to a licensing agreement with the American NCR Corporation. [For more on the early Elliott range, click here or see Gabriel (1997/2003 online), or Hoare (1981/2003 online). Jack Smith (2001/2003 online) provides a handy memoir of how an Elliott 403 contributed to the Australian weapons research programme. For the later "800-Series" machines, see Part 5.]

�

Key Innovation - Direct Memory Access: The architecture devised for the Elliott 405 allowed its programmers to maximise CPU performance by re-routing much of its non-essential traffic around it rather than through it. Using an electronic equivalent of a city orbital [US = beltway], data en route from peripheral to Main Memory was given a direct access route, without going via the CPU (hence the epithet "direct"). This was implemented by giving each peripheral its own controller, with its own instruction set. This way of doing things immediately became an industry standard, and is nowadays known as "Direct Memory Access" (DMA) [further details]. << AUTHOR'S NOTE: The DMA set-up is not in essence dissimilar to the psychological concept of "implicit learning", that is to say learning which gets stored without ever having been consciously attended to, however, our full evaluation of this possibility will not be available until 2004. >>

�

The success of the 400-series kept Elliott so busy during the early 1950s that they even had to give some of their good ideas away. We refer here to the 1952 "Nicholas" - another Charles Owen design, with system software by George Felton - a machine which Elliott put together in a hurry to carry out in-house data processing. So successful did this prove, that the NRDC arranged for Ferranti's London works to turn it into a marketable package, and the result - the "Pegasus" - ran its first program in October 1955 and entered commercial service six months later (Lavington, 2000). It proved a highly reliable general purpose serial computer, and despite having to make do with thermionic valves and magnetic drum storage, its designers had devised a "powerful, straightforward" (p16) 49-item three-operand instruction set which made for very concise programming, although in achieving this they had been assisted by one of the NRDC engineers, Christopher Strachey .....

�

BIOGRAPHICAL ASIDE - CHRISTOPHER STRACHEY: [This from Campbell-Kelly's (1985) technical biography.] Christopher Strachey (1916-1975) was the son of a First World War cryptologist. He graduated in mathematics and physics at Cambridge, and ended up as a wartime radar engineer. Employed in 1939 by the Standard Telephone and Cable Company on the development of thermionic valve technology, he had occasion to run complex calculations through a small differential analyser, and this aroused his interest in computing. Upon his demobilisation in 1945, he obtained a position as a science teacher, firstly at St. Edmund's School, Canterbury, and later at the prestigious Harrow School, where a chance encounter with Norbert Wiener's recently published "Cybernetics" re-awakened his interest in computing. He therefore approached the National Physical Laboratory's Pilot ACE development team a few miles away across Middlesex, and by January 1951 was attempting to write programs in his spare time. Inspired by an article on computerised board games by the NPL's Donald Davies (Davies, 1950), he chose to attack the problem of getting a computing machine to play draughts. He did not quite succeed, but his program was promising enough to earn him a second attempt, this time on the Ferranti Mark I at Manchester University, and over the ensuing months he improved the code so much that the university promised him a job as soon as finances allowed. In the event, however, he was poached by Lord Halsbury's newly created NRDC before this could happen, and joined them in June 1952. He presented his draughts program at the Second ACM Conference in Toronto in September 1952, and the consequent boost to his reputation then earned him a succession of roving placements in both North America and Britain. This included time on site in 1953 evaluating progress of the Elliott 401, in 1954 improving the Ferranti Pegasus, and in 1958 helping to develop the MUSE [more on which in Part 5]. He left the NRDC in 1959 to set up in private consultancy, and for six years toured the country, contributing to the development of third generation programming languages [see Part 6]. Finally, he was headhunted in 1965 by Oxford University to direct their recently established Programming Research Group, and�worked there until his death in 1975.

�

One of the things Strachey did for Pegasus was to help design an enlarged set of CPU registers, in what is now known as a "general register" arrangement (Lavington, 2000). He began by looking at the inefficiencies and inadequacies of earlier instruction sets and found that the chief time stealers - the "red tape" instructions - were control branches, data transfers, and handling large data arrays. He therefore arranged a simultaneous upgrade of the logic circuitry, the register set, and the instruction set. A 39-bit word size was chosen, within which was a six-bit op code field. This would theoretically have allowed up to 64 separate operations, but only 49 were initially used. The operations were presented as eight blocks of eight, by sub-dividing the op code into two three-bit triplets, and transliterating each of these into octal. Thus 010011 was treated as 010 (octal/decimal = 2) followed by 011 (octal/decimal = 3), and shown as 23 (decimal 19).

�

TECHNICAL ASIDE - OCTAL: Octal is the base eight numbering system, which means that each left order integer represents a power of eight higher than the one to its right. Only the digits 0 to 7 inclusive are used, because the decimal 8 will always "carry one" - thus octal 7 plus octal 1 gives octal 10 (decimal 8), and octal 777 (decimal 511) plus octal 1 gives octal 1000 (decimal 512).

�

Programmers soon began to remember these numeric codes as "mnemonics" for the operations they controlled [more on which in Section 3.2]. Eight accumulator registers were built into the CPU, numbered X0 to X7, with X0 always set to binary zero. These could be used for any relevant purpose, specifically indexing large data arrays, loop counting, and holding intermediate values during complex calculations. This is how array handling worked .....

�

Key Innovation - Handling Data Arrays: When allocating Main Memory for data storage, it frequently happens that a data item naturally occurs a number of times. Suppose, for example, that you needed to store <GROSS-WEEKLY-PAY> for every one of a thousand employees. The trick here is to set up a contiguous "data array", that is to say, one thousand consecutive word addresses in Main Memory. You can then select the particular occurrence you are momentarily interested in by "indexing" that array with a bracketed suffix. Thus <GROSS-WEEKLY-PAY(INDEX)> will get you record #8 when INDEX reads 7 (all computer counts start from zero, remember), record #652 when it reads 651, and so on (it will also blue screen on you if you are careless with your indexing and try to access a value less than zero or greater than 999). << AUTHOR'S NOTE: Neuroscience does not know enough about biological processing architectures to judge whether the brain has devised some sort of array indexing system for itself. We suspect that it does, but the full evaluation will not be available until 2004. >>

�

About 40 Pegasus machines were sold before the product line was discontinued in 1962: the sixth off the production line helped Vickers Armstrong design the Concorde supersonic airliner, and the 25th has been restored to working order and is on display in the Science Museum, London. "On the international stage," writes Lavington (2000, p57), "the huge number of modern commodity computers with a general register-set architecture is testimony to the usefulness of Christopher Strachey's ideas for Pegasus". [For background detail on Pegasus, see Cooper (1990/2003 online) or Lavington (2000). Donald Kershaw was a user of this kit, and provides some recollections online (Kershaw, 2002 online).]

�

2.4 EDSAC and EDSAC Clones

We have already explained in Part 3 how Cambridge University's EDSAC had come from behind in the years 1946 to 1949 to put together the world first stored program computer [reminder]. The machine continued to be improved during the early 1950s, and brought forth many innovations, not least microprogramming [more on which in Part 5] and assembler programming [see Section 3.2 below]. For Cambridge University's official collection of reminiscences from this period, click here.

�

The EDSAC development which most concerns us at this juncture is Maurice Wilkes', David Wheeler's, and Stan Gill's work on "subroutines", ca 1947. Smotherman (2002/2003 online) reviews the early history of the subroutine concept, and suggests that Ada, Countess of Lovelace, deserves first mention in that in 1842 she proposed reusable routines for Babbage's Analytical Engine [reminder]. Alan Turing then proposed much the same for the Pilot ACE in 1946, followed by the EDSAC team in 1947. Konrad Zuse also deserves mention in that his Z4 was upgraded ca. 1950 to allow a single subroutine to be executed by mounting it on a second tape reader. Here are the some of the technicalities .....

�

Key Innovation - Subroutines: Subroutines are "self-contained sequences of orders each of which performs a distinct part of the calculation in hand" (Wilkes, 1956, p86). More importantly, they allow a discrete operation to be developed once and then re-used as often as wanted, and as such they represent probably the greatest single aid to programmer productivity ever devised. In practice, however, there are several ways of benefiting from pre-written code. The first is to copy a trusted pre-existing program in its entirety and then edit it where necessary, the second is to edit in a block of instructions from such a program, and the third is to set a block of instructions up as a program-in-miniature - the "subroutine" - and then formally transfer control to it. All three ways work, but only the subroutine approach creates genuinely reusable code. The transfer of control is known as making a "call", and the eventual transfer back is known as a "return". Wilkes explains that it is frequently an engineering choice whether to perform a given operation via a subroutine, or else to hardwire in some additional logic circuitry. As is often the case in such matters, there is a trade off between complexity (circuitry costs money to design and test) and speed (subroutines add execution time), and it is the relative frequency of the task which often swings the decision one way or the other. The EDSAC, for example, deliberately chose a slow subroutine to perform division on the grounds that divisions were comparatively infrequent. Other "obvious examples of subroutines suitable for including in a library are ones for calculating sines, cosines, exponentials, and similar functions" (ibid.). << AUTHOR'S NOTE: Biological cognition probably relies heavily on subroutine calls. Taking speech production as an example, and adopting the hierarchical view proposed by psycholinguists such as the University of Arizona's Merrill Garrett (see, for example, Garrett, 1990), we find that there are several points at which the primary justification for the existence of a high level process can only be to control the rapid concatenation of a sequence of lower level ones. Thus with the allocation of phonemes to words at the phonological build stage, or with the allocation of muscle contraction profiles to phonemes at the subsequent muscle activation stage. Another (very significant) property of subroutines is covered when we get to the work of John McCarthy in Section 4.7. >>

�

Key Innovation - "Return Codes": One of the complications introduced by subroutines is that the calling program must be sensitive to the possibility that the called subroutine might fail in some way. Such a failure will often "crash" or "loop" the entire machine, but if it does not, then it falls to the calling program NOT to carry on in what is then probably a corrupt state. The way around this is (a) for the calling program to precede the call with the setting of a "return code" data field to a non-zero value (to be interpreted as "failure"), (b) for the last instruction in the called program to set that same field to zero (to be interpreted as "success"), and (c) for the first instruction in the calling program after the return to test for zero or non-zero. If the subroutine has worked, then the return code will have been set to zero, and the calling program can proceed about its business. If the subroutine has not worked, then the calling program can branch to a pre-written soft abort control sequence. So successful did the return code idea work in practice, that designers soon introduced a whole range of non-zero return codes (alternatively, "error codes" or "diagnostics"), each enabling a different type of failure to be accurately identified. Internet users see these every day, of course - click here for a non-existent Internet link, and note the resulting error code. << AUTHOR'S NOTE: Neuroscience does not know enough about biological processing architectures to judge whether nature has invented for itself some sort of cognitive return code. For our part, we suspect not only that it has, but also that the mechanism works closely with the biological interrupt system [see Section 1.3] to support hierarchical control in general terms. The issue will be reviewed in greater detail in Part 7. >>

�

EDSAC was also the inspiration for the Lyons Company's LEO I [reminder], and here again the main player was a demobilised wartime boffin, an ex-TRE electronics engineer named John M.M. Pinkerton (1919-1997). Pinkerton took charge of the project in January 1949, and the resulting EDSAC clone ran its first test program on 5th September 1951 (Bird, 1994), and its first business application - a bakeries valuation - on 29th November 1951 (Land, 1999/2003 online). It then parallel ran its first live payroll on 24th December 1953, at which time its designers claimed that what took an experienced clerk eight minutes could be done by the machine in 1.5 seconds (Bird, 2002). After careful cross-checking of manual and computer output, the manual system was withdrawn, and LEO ran unsupported on 12th February 1954 (Caminer and Land, 2003). Apart from the immediate benefit to their Accounts Department, Caminer and Land (2003) describe that date as marking "a world first". A more powerful machine, the LEO II was begun in 1954 and entered service in May 1957. Eleven of these machines were built, but only the last four were equipped with the more expensive ferrite core memory (Ferry, 2003). A third generation machine, the LEO III is mentioned in Part 5.

�

�

3 - First Generation Software�

Alongside the electronics and engineering developments, the early 1950s also found a growing army of computer programmers struggling to keep their CPUs gainfully employed. The problem here was essentially one of too good a press, for now that potential system sponsors knew how versatile the new digital machinery could be, they were raising requests for computer solutions faster than the necessary programs could be written. Unfortunately, the penny was also beginning to drop as to how much time these programs could take to write; and worse, how difficult they then were to amend; and worse still, how often customers would ask for such amendments. Thus .....�

�

"In June 1949 I realised that a good part of the rest of my life would be spent finding errors in my own programs." (EDSAC's Maurice Wilkes, in Sabbagh, 1999.) [We believe (May 2003) that Wilkes is still alive, so it is clearly good to make lots of errors!]

�

3.1 Application vs Operating Software

To cope with this early "software backlog", the computing industry was forced to specialise on the programming side of things the way it had specialised during its formative years in the physics and the engineering, when valve men with soldering irons produced the hardware and mathematicians decided how it should be used. It began holding summer schools (for example at Cambridge in 1950) and providing training courses, and at first two, and then three, and then four, areas of specific expertise emerged. The first distinction to emerge was that between "computer hardware" - the physical machine - and "computer software" - the programs which had to be written to make the hardware productive [the word "software" itself was coined by John Tukey at Bell Labs]. The second distinction was then between "application software", programs which specifically addressed this or that real world problem, and "operating software" (sometimes "systems software"), programs which helped interface these applications with the hardware in some standardised and user-friendly way. A computer's "Operating System", or "OS", became the software which was actually "closest to" the hardware, and what was happening in essence was that one program - the OS - ended up (thanks to yet more call-and-return branching) executing another - the application .....

�

Key Innovation - Application Software: Applications are what you, the user, want to do with your computer. If you wanted to run an appointments system, for example, you would need a carefully designed and interlocking set of "applications", each addressing one aspect of the overall task of managing appointments. The distinct practical purpose of each program is known as its "functionality", and it is functionality which gives an application its business value. The sum total of individual applications is known as the "application system", and the programmers who specialise in coding applications are known as "applications programmers". Applications programmers are assisted in their work by the "programming languages" available on their machines [see next section], and also by a deep understanding of the real world they are trying to computerise. In the event, the best programmers often turned out to be the worst at understanding the real world, and so a separate breed of beings called "systems analysts" was recruited to do that part of the job for them, and hybrids of the two sorts of skills - those who can manage both real world and programming tasks - are called "analyst-programmers".

�

Key Innovation - Operating Software: The key to understanding operating software lies in grasping a single basic fact, namely that a high proportion of any one program re-uses code already written (often at great cost) elsewhere. It therefore makes sense not just to provide the applications programmers with a library of tried and tested subroutines, but also with a way of accessing these procedures from within a programmer-friendly keying environment. The operating system is the mechanism by which this is all achieved, and its standard functions include interfacing with the operator, managing the library procedures, managing the peripherals, displaying the contents of magnetic media, knowing what filenames have been allocated, knowing where on the available disks it has put each file (and the really clever operating systems are able to store a given file in a number of non-contiguous fragments, so that takes a lot of managing), managing disk I-O, issuing warnings, help, and error messages, keeping track of date and time, and marking same on the filestock so users know how old they are, and last (but by no means least) controlling the execution of end-user programs. Some operating systems are easier to get along with than others. The worst ones are those invented in the days before the general public were allowed to get their hands on computers. The easy to use ones are called "user friendly", and the trend of late has been for them to become more and more user friendly as time goes by.��Operating systems are written by boffins and maintained by "systems programmers", who (because they are the only ones who understand what they are doing) get paid a lot more than applications programmers.

�

Figure 1 shows where all these (very expensive) people fit within a commercial computing installation .....

�

Figure 1 - Key Roles in Corporate Computing: Here we see the basic structures and personalities you would find in a typical computerised corporation. Two separate organisations are shown. On the left hand side of the diagram is a computer manufacturing organisation (Ferranti, say). On the right hand side of the diagram is a client organisation, a purchaser-user of computing power. We have subdivided the client organisation so as to show its Computing Division (large panel, left, drawn over scale) separate from its end-user divisions (stacked panels, right, drawn under scale), and both subordinate to the Corporate Executive (top centre). Within the Computing Division, we show Operations and Systems Development activities separately, albeit both share a common physical system (pink panel). Within the Systems Development Department, the principal activity is the writing and testing of application software for the future, whilst within the Operations Department it is running application software already formally authorised for live use. This separation of duties is vitally important. Indeed in most large organisations, the operations side of things will be located in a purpose-built "Computer Centre", whilst the development side of things remains a headquarters function. � Note the many different specialists (red captions) dotted around the various departments. In the manufacturing organisation, we find the boffins who build the hardware and develop the operating software. Hardware boffins improve circuitry and invent memory mechanisms, etc. Software boffins improve instruction sets, implement ever better pipelining, etc. "Maintenance engineers" provide an agreed level of after-sales service (green arrow) in return for a consideration (pink arrow). In the Operations Department, there are "operators" to tend the machine itself and "systems programmers" to keep it in tune. In the Systems Development Department, it is usual to find a number of "systems maintenance teams", each consisting of the analysts and programmers familiar with a particular application system. There will usually also be an overarching "Systems Unit" [names vary] carrying out more strategic investigation and design. � Given the many specialisms,�the need also arises for help desks and liaison officers (green captions). These include "operations support" people to liaise with the systems maintenance people in order (a) to schedule work on a day-to-day basis, and (b) to report and prioritise system errors. The maintenance team also have to liaise with the Systems Unit over the commissioning of broader system enhancements. [For the sort of systems disasters which result when people do not communicate effectively, see our IT disasters database.]

<![if !vml]><![endif]> � Copyright � 2003, Derek J. Smith.

�

3.2 Interpreters, Assemblers, Loaders and Linkers, and Compilers

As far as software productivity is concerned, there has only ever been one problem and there has only ever been one solution to it. The problem is the unfriendliness of machine code, and the solution has been to distance programmers from its complexity as far as possible. Put simply, writing efficient machine code from scratch is onerous in the extreme, and trying to work out how you did what you did at a later date is nigh on impossible. As a result, generation zero development teams devoted a lot of effort to simplifying the process, and their answer was to write utility programs to stand between the programmers and the machine code. In their simplest form, such programs offered to translate from a relatively human-friendly vocabulary into the decidedly human-unfriendly vocabulary of the machine's instruction set. These utilities were the first "programming languages", and they brought with them some important new terminology .....

�

Key Innovation - Programming Languages, "Source Code", and "Object Code": A programming language is a computer program capable of recognising selected human-meaningful input commands, and generating as output the logically corresponding machine code instructions. The input commands are known as the "source code", and source code - of essence - is far easier to read for its logical intention than is machine code, both by the original programmer during development, and then by a maintenance programmer at a later date. The output is known as the "object code", and if the developers have done their job properly this will have been optimised for technical efficiency, as well as being standardised and bug-free. Special dummy source code instructions allow explanatory annotation to be added to the source program as aides memoires, but withheld from the machine code.

�

Ideally, you want to be able to tell the machine something you understand, like <MULTIPLY TAXABLE-PAY-THIS-WEEK BY TAX-RATE GIVING TAX-THIS-WEEK> and have it turn that instruction into something the machine's arithmetic unit will understand, like <00101111001011011000101101001110110111 etc.>. Unfortunately, the source instructions had to be kept short to save memory, and so were usually abbreviated to two- or three-character "mnemonics" such as M for MULTIPLY .....

�

Key Innovation - Mnemonics: In everyday English, a mnemonic is a memory aid of some sort (such as "ROY G BIV" for the colours of the rainbow, etc.). Because there is no point in having source code at all unless it is distinctly memorable, mnemonic systems usually consist of a set of handy verbal tags .....

�

Example: We have already seen how Pegasus programmers took to identifying their binary op codes by their octal couplets. Thus Pegasus op code mnemonic "four-one" [see Section 2.3] instructs the machine to add a particular given number to a particular accumulator. Not only is "four-one" easier to remember than the logically identical binary code "one-zero-zero-zero-zero-one", but it is also easier to convert back to binary than the decimal equivalent.�Thus octal 4 gives you binary 100 and octal 1 gives you binary 001, so octal four-one gives you 100001.

�

The simplest form of translation utility works on a one-to-one basis, and would nowadays be termed an "Interpreter". However, this level of assistance was not a lot of use to generation zero machine code experts, and so was little used. Instead, the boffins concentrated on a more powerful class of utilities known as "assembly languages", or "assemblers" for short .....

�

Key Innovation - Assembly, Loading, and Linking: "An assembler is a translator that translates source instructions (in symbolic language) into target instructions (in machine language), on a one to one basis" (Salomon, 1993/2003 online). Jack Gilmore wrote such a program for the Whirlwind, using mnemonic codes, and allowing decimal input. The Assembly process then substituted one or more machine instructions for the mnemonic and turned the decimals into binary.�This was followed by SHORT-CODE, developed by John Mauchley, and A-0 developed by Grace Hopper. Wilkes (1956) claims "an assembly subroutine" for the EDSAC in 1950. << AUTHOR'S NOTE: Neuroscience does not know enough about biological processing architectures to judge whether the brain processes some sort of machine code in some locations and some sort of assembler mnemonics in others. What can be said with certainty is that there are repeated major shifts in data coding as information flows end-to-end through the cognitive system. The most thorough illustration of this we have come across is Ellis's (1982) transcoding diagram, in which each separate information flow is carefully tagged with the nature of the coding method used within it. >>

�

One point about simple assemblers was that the object code they produced was actually incomplete. Specifically, it did not include the object code for library-held subroutines. An additional processing stage was therefore needed, called "loading and linking". This was/is an additional utility program which located the various object code modules and - after some careful address cross-calculations - concatenated them into a single executable object code program. When done properly, all subroutine calls will branch and return to valid addresses. �

�

Further major weaknesses gradually became apparent. Firstly, because machine instructions are individually very simple actions, there often needs to be a one-to-many relationship between a mnemonic and its equivalent object code instructions. Secondly, the machine code from one source instruction often needs interleaving with the machine code from adjacent source instructions. For example, mnemonic-by-mnemonic execution may be ideal for interacting with a human operator, but it is totally unable to cope with intentional program loops. And thirdly, assemblers were hardware-specific. You could not write source code once, and have it assembled onto different computers. This is because the one-to-one relationship between source and object instruction tied the translation program inflexibly to the specific host's specific instruction set. This led to the development of more advanced, machine-independent, programming utilities called "compilers" .....

�

Key Innovation - Compilers: A "compiler" is a package of utility programs, fronted by a powerful conversion routine, supported by all the necessary loaders and linkers, and complete with copious error detection and optimisation routines. As such, they allow programmers to write neatly structured source code, relying heavily on subroutines, and yet leave all the technical details to the compilation process. Where both the amount and the complexity of the object code per source instruction is high, the language is known as a "high level language". In 1951, R.A. ("Tony") Brooker was put in charge of Manchester Mark 1 software and by 1954 had produced the first recognisable compiler, namely the "Mark 1 Autocode". Compilers can conveniently be divided into "scientific", where there the source approximates to formal mathematical representation, or "business-oriented", where the source approximates to everyday English. The standard example of the former is FORTRAN, and of the latter, COBOL. The way things timed out, many first and second generation business systems were written at great cost in Assembler, only to find themselves "up for a re-write" ten years later when COBOL came along. In investment terms, this is not good commercial practice, and British industry (at least) was so eager to squeeze every drop of return out of its capital, that commercial systems written in Assemblers during the early 1960s were still around (as "legacy systems") nearly 30 years later. The present author did a stint on a large Assembler maintenance project as late as 1988.

�

In fact, the word "compiler" should be treated with some caution, because its meaning drifted somewhat in the early years. Backus (1984) argues that many early "compilers" are better regarded as assemblers, for their most significant aspect is their mnemonic set, rather than any program structuring functionality, while Hopper (1959) emphasises their role in managing subroutines. We include a full (albeit short) Assembly program, and elements of a compiled program, in Part 6, where we shall also be looking at how programming languages give programmers hands-on control over their machine's short term memory resources.

�

4 - First Generation Artificial Intelligence

Thus far, we have limited our consideration of software to military applications such as codebreaking and ballistics, and to civilian applications such as payroll. However, a number of more adventurous applications had been bubbling under during the late 1940s, and finally hit the headlines in the early 1950s. These were (1) machine translation, (2) semantic networks, (3) mechanised game playing, (4) man-machine conversation, (5) "neurodal" networks, (6) robotics, (7) machine problem solving, and (8) artificial consciousness. Taken together, these make up a domain of enquiry we know today as "artificial intelligence" (AI), and in the remainder of this section we look at each of these AI traditions in turn .....

�

4.1 Machine Translation (MT)

Hutchins (1997/2003 online) dates the idea that computers might be used to automate the process of natural language translation to two conversations which took place 20th June 1946 and 6th March 1947 between the aforementioned Andrew D. Booth [see Section 2.1] and Warren Weaver (1894-1978), of the Rockefeller Foundation, and a letter dated 4th March 1947 from Weaver to the cyberneticist Norbert Wiener .....

�

BIOGRAPHICAL ASIDE - WARREN WEAVER: �Weaver was a University of Wisconsin mathematics professor, turned career philanthropist, turned wartime government advisor, turned "wartime 'Mr Fix-It'". He joined the Rockefeller Foundation in 1932 as Director of the Division of Natural Sciences, did much of the facilitation during the early construction of the giant 200-inch Hale Telescope at the Mount Palomar Observatory [this project actually lasted 1928 to 1948, with a break during World War Two - click here for the official history], and by the start of the war had established enough of a reputation as a loyal scientist to be appointed Director of the Control Systems Division of the newly formed US National Defence Research Committee (NDRC) [not to be confused with the British NRDC mentioned in Section 2.2]. Much like Princeton's John von Neumann [reminder], his role within the NDRC made him a roving advisor-facilitator with responsibility for just about every mathematical aspect of the US war effort [mostly classified, of course], not least the transition from analog to digital computation (see Mindell, 2002). It also involved him in at least one top secret mission across the Atlantic. After the war, Weaver became interested in the concept of MT, and it was on one of his convenient Rockefeller Foundation visits that he met Booth, who, as we are about to see, shared his belief that MT was theoretically possible. He also went official second author on one of the major classics of modern information theory, namely Shannon and Weaver (1949).

�

Following Weaver's inspired and energetic string-pulling, one of the first computers off the blocks was Booth's own ARC [see Section 2.1], and the formative event here seems to have been a chance professional exchange of minds between Booth and the Cambridge geneticist Richard H. Richens. Richens dates their first meeting to 11th November 1947, and explains that it came about thanks to their shared interest in punched card methods of processing research data. Richens later explained that .....

�

"..... the idea of using punched cards for automatic translation arose as a spin-off, fuelled by my realisation as editor of [the journal Plant Breeding Abstracts] that linguists conversant with the grammar of a foreign language and ignorant of the subject matter provided much worse translations that scientists conversant with the subject matter but hazy about the grammar." (Richens, 1984; private correspondence cited in Hutchins 2000)

�

The unlikely pairing of the x-ray crystallographer with the drum storage system and the geneticist with punched card experience then set to work trying to mechanise translation using a stored dictionary. In a jointly authored paper published a few years afterwards, they explained what progress they had made so far, and made a number of predictions as to how they saw the new science developing .....

�

"The most obvious way of using a computing machine for translation is to code each letter of a proposed dictionary word in 5-binary-digit form and to store the translation in that memory location having the same digital value as the aggregate value of the dictionary word. Translation would then consist in coding the message word (a teletyper does this automatically) and then extracting the translation in the same or next lower (digitally valued) storage position, the first giving exact translation and the second the stem translation with a remainder. [.....] "

�

TECHNICAL ASIDE: This is our old friend the Baudot telegraph code again [reminder; details], in which the letters of the alphabet are given binary codes in the range 00000 (decimal 0) to 11111 (decimal 31). The letters C-O-W, for example, would be given the following codes .....

�

C = 01110 (decimal 14)

O = 11000 (decimal 24)

W = 10011 (decimal 19)

�

What the authors were suggesting was that they should concatenate the three separate five-bit binary codes to give a 15-bit binary code (in this case, 011101100010011 or decimal 15,123). This code would then be automatically generated by typing the word "COW" at any standard teletype keyboard, and all that the computer needed to do was to store the translation of that word at storage address 15,123 in the computer's memory. In practice, however, it was not that easy .....

�

"Despite its simplicity, this scheme is quite impracticable with any existing storage device because the naive process of placing each dictionary word in a storage position equivalent to its own binary numerical version is grossly redundant."

�

TECHNICAL ASIDE: The San Jose Scrabble Club lists 973 three-letter words in the English language, whilst there are 32,768 different 15-bit addresses between 000000000000000 and 111111111111111 inclusive. Storing one in the other would therefore give you an unused memory factor of approximately 97%! This would be an unforgivable waste of resource even in the modern world of cheap memory, but in the 1940s would have been a total non-starter.

�

"[..... so] to overcome this difficulty, a slightly more sophisticated approach has to be adopted. Dictionary words are stored in alphabetical order and in consecutive memory locations. Possibly these might be broken up into initial letter groups such that each group starts in a storage location indicated by the binary coded form of its initial [.....] The coded word to be translated is sent to an arithmetic register and is there compared, by subtraction, with the dictionary words. In this way, by means of the normal conditional control transfer order [ie. you only do it when the aforementioned subtraction gives zero, which will only happen when the two items are identical - Ed.], the dictionary stem corresponding to the word can be obtained and its translation printed. Next, the remainder of the message word, left after subtraction of the stem, is shifted to occupy the extreme left-hand register position, and the comparison process is repeated, this time with the words of the ending dictionary." (Richens and Booth, 1955, pp45-46)

�

TECHNICAL ASIDE: The authors are here addressing the problem of translating morphologically complex words such as "COWS", where we have a singular root noun, referred to here as the "stem", to which is suffixed the pluralising morpheme "-S". This is the problem of "inflection" .....

�

BASIC LINGUISTICS - INFLECTION: "Inflection is variation in the form of a word, typically by means of an affix, that expresses a grammatical contrast which is obligatory for the stem's word class in some given grammatical context" [source]. It is the mechanism by which many of the world's natural languages cope with the number (single or plural), gender (masculine, feminine, or neuter), and grammatical role of adjectives and nouns, and the number (single or plural), tense (past, present, or future), and person (first, second, or third) of verbs. Thus the suffix <s> will pluralise many English nouns (eg. "cats") or put many verbs into the third person singular (eg. "he eats"). �

�

Richens' and Booth's proposal was that such words should be matched in two stages, from two functionally separate dictionaries, namely a stem dictionary and an ending dictionary. Thus COWS would divide as <COW + S>, translate into German as <KUH + E> and be reconstituted (on this occasion correctly) as KUHE. In practice, however, this was not that easy either .....

�

Other universities quick to get involved were MIT and the Harvard Computation Laboratory. The first MT conference was organised by Yehoshua Bar-Hillel of the University of Jerusalem (fortuitously on a sabbatical at MIT), and took place 17-20th June 1952. Among the delegates were Whirlwind's Jay Forrester, Harry Huskey, Andrew Booth, and the linguist in charge of the simultaneous translation facility at the 1945-1946 Nuremberg Trials, Georgetown University's Leon Dostert. This conference was followed two years later by the "Georgetown-IBM" demonstration of automated translation. This was organised by Dostert, took place on 7th January 1954 at IBM Headquarters, New York, and used software developed by Paul L. Garvin with technical input from IBM's Peter Sheridan and management backing from Cuthbert Hurd [whom we have already met in Section 1, and of whom even more in Part 5]. The demonstration processed a 250-word Russian-English vocabulary, and made front page news the next morning [see the New York Times article; see the IBM Press Release].

�

BIOGRAPHICAL ASIDE - ANTHONY G. OETTINGER (1929-): Anthony G. Oettinger (1929-) studied for his 1954 doctorate at the Harvard Computation Laboratory on the requirements of an automated Russian-English dictionary (Oettinger, 1955). Given the abysmal state of US-Russian relations in the McCarthy era, there is little doubt that it was the promise of being able to translate intelligence material out of Russian more effectively which made MT a defence issue, and which therefore secured it much of its funding [see, for example, Bar-Hillel (1960)]. We meet Oettinger again in Section 4.5.

�

Unfortunately, the Richens-Booth approach only works for the relatively simple words in a language, and starts to descend into a morass of ever more complicated logic branches and special rules once it encounters "irregular" words and complex grammatical structures. Erwin Reifler, of the University of Washington, Seattle, was one of the first to try to design his way around this particular problem. In Reifler (1955), he proposed carefully allocating separate dictionaries to separate "operational form classes", that is to say, to each of the standard grammatical categories. Form classes with very large memberships include nouns (cup, table, man, etc.), adjectives (big, happy, etc.), and verbs (eat, walk, fly, etc.), and form classes with very small memberships include determiners (a, the, some, etc.), pronouns (I, she, it, etc.), auxiliary verbs (to be/have/get, etc.), prepositions (in, to, by, etc.), and conjunctions (and, but, because, etc.). Clearly recognising the modular nature of biological language processing, he then proposed separate drum memories - up to 15 of them! Thus .....

�

"If magnetic drums are used for memory storage, we may distinguish between large-drum memories and small-drum memories. A number of each will be needed. [.....] Operational form classes whose membership is large will require large drum memories. [//] Large-Drum System: [.....] For the large operational form classes, a total of four large-drum units will be needed, containing the following memories: (a) The capital memory for substantives and substantive constituents, (b) The attributive adjective memory, including the cardinal numerals, except the few included in the memory of determinative 'pro-adjectives', (c) The principal verb memory, excluding the few verbs included in the memory of verbs always or sometimes requiring a predicate complement, [and] (d) The predicate adjective memory, including all adverbs of adjectival and numerical origin. Small-Drum System: In the small-drum system, an individual memory will be assigned to each of the operational form classes with a comparatively small membership. The ten small operational form classes discussed above may be entered into ten small drums, but it may be desirable with German to subdivide the class of propositions into six sub form-classes, according to whether they require their complements in the following cases: genitive, dative, accusative, genitive and dative, genitive and accusative, or dative and accusative. This would increase the number of small drums to 15. With the smaller number of drums, a greater complexity of equipment and circuitry would be necessary to handle the prepositions. It is, thus, a matter of engineering economics whether 10 or 15 small-drum memories are used." (Reifler, 1955, pp159-160; italics original)

�

TECHNICAL ASIDE: In retrospect, Reifler's suggestion is nowhere near as silly as it might sound; it was just a couple of years ahead of its time. With the development of database technology over the ensuing decades, it gradually became possible to store fundamentally different datasets such as Reifler was proposing on a single physical resource. In other words, any Database Management System (DBMS) worth its salt could easily make a single magnetic disk look like 15 disks. Figure 2(b) shows the sort of data analysis which would have needed to be done in order to allow this to happen. We shall be saying a lot more about how DBMS do what they do in Part 5 (Section 3.1) and Part 6, and about why they are so important in Part 7.

�

Coping with noun plurals and (in certain languages) case variations, as well as with verb endings, was, however, just the beginning, for it soon became apparent how complicated were the real-life languages programmers were trying to translate. The problems posed by irregular verbs, or - worse - by the use of ironical or figurative language, or by the intricacies of pronoun resolution, or by the effects of context, or by "polysemy" (that is to say, the multiple meanings of words), have still not been overcome half a century later, and the 1950s efforts were often laughable. Here, from Masterman (1967, p202), is an example of just how hopeless simple word-for-word translation can be [compare (2) and (3) for intelligibility, and note that Latin is a heavily inflected language] .....

�

(1) ORIGINAL LATIN: Possibile est, at non expertum, omnes species eiusdem generis ab eadem specie ortum traxisse.

�

(2) WORD-FOR-WORD MACHINE TRANSLATION: Possible is however not prove/lacking all species/appearance same genus/son-in-law from same species/appearance arise draw.

�

NB: Note the underlined alternatives here. These are examples of the multiple meanings mentioned above, where a single L1 word has two or more valid L2 translations, and where the final choice relies on contextual cues.

�

(3) FINAL ENGLISH: It is possible, though not proved, that all species of the same genus have been derived from the same species.

�

Now Margaret Masterman (1910-1986) knew Richens thanks to their joint involvement with Cambridge University's Language Research Unit .....

�

"Research on MT at Cambridge began [.....] with the informal meetings of the Cambridge Language Research Group in 1954 [.....] In 1956 a grant was received from the National Science Foundation to pursue MT research, and the Cambridge Language Research Unit (CLRU) was formed, a research organisation independent of the University of Cambridge, with Margaret Masterman [wife of the philosopher R.B. Braithwaite] as director. In later years research grants were also received from the US Air Force Office of Scientific Research, the Office of Scientific and Technical Information (London), the Canadian National Research Council, and the Office of Naval Research (Washington, DC) [.....] By 1967, active research on MT at CLRU had declined; many of its members had moved elsewhere, mainly within the Artificial Intelligence field and often with a continued interest in MT [.....] There were four main themes in its MT research: the thesaurus approach, the concept of an interlingua, 'pidgin' translation, and lattice theory. The focus was primarily on semantic problems of MT, and syntactic questions were treated secondarily. Although procedures were intended to be suitable for computers, most of the proposals were tested only by manual or punched card simulations because access to a computer proved to be difficult for many years." Hutchins (1986/2003 online)

�

All four CLRU avenues of attack are directly relevant to our argument, including their investigations of pidgin. Indeed, one of the group's most fruitful observations was that there were some rather telling similarities between the halting and generally awkward word-for-word machine translations [sentence (2) above] and real life "pidgins" .....

�

BASIC LINGUISTICS - PIDGINS AND CREOLES: A "pidgin" is "a reduced language resulting from contact between groups with no common language", and a "creole" is "a pidgin or jargon that has become the native language of an entire speech community, often as a result of slavery or other population displacements" [source]. Thus Pidgin English is any rudimentary form of English, especially if spoken by aboriginal peoples for trading purposes, and the pidgin spoken in Papua New Guinea contains many quaint phonetic corruptions, such as lusim (= abandon), bigpela man (= adult), klostu (= almost), and narapela (= same again) [to see the full dictionary, click here].

�

The academic point was that since pidgins somehow contained all that was necessary to render a complex message meaningful across a language barrier, it might make sense to design some form of�"mechanical pidgin" into your machine translation system. And if you did this carefully, you would then be able to record most of L1's inflections in coded form, tucked in as "pidgin markers" alongside the word stems, giving you what is known as a "continuous form" translation. With a continuous form version safely under your belt, you would have a much better chance of producing a contextually, as well as grammatically, sound L2 draft. According to Masterman (1967), the term "mechanical pidgin" came from Richens. Here, still from Masterman (1967, p202), is how sentence (2) above might look in continuous form, complete with inset markers .....�

�

(4) CONTINUOUS FORM MACHINE TRANSLATION WITH INFLECTION MARKERS: Possible z is however not prove/lacking z all m species/appearance same g genus/son-in-law z from same species/appearance o arise z draw p.

�

Masterman selected her inflection markers from the following list from Richens and Booth (1955, p35) .....

�

a = accusative case; d = dative case; f = future tense; g = genitive case; i = indicative mood; l = locative case; m = multiple; n = nominative case; o = oblique; p = past tense; q = passive form; r = partitive; s = subjunctive mood; u = untranslatable; v = vacuous; z = unspecific

�

The technical name for an intermediate translation form of this sort is an "interlingua" .....

�

Key Innovation - Interlinguas: An "interlingua" is a language form intentionally divorced from both L1 and L2 in a translation, but from which both L1 and L2 (and, ideally, all other languages) can be generated. Strictly speaking, this definition incorporates international languages such as Esperanto, but we restrict ourselves here to any sequence of heavily inflected protosemantic codes produced by parsing. << AUTHOR'S NOTE: Neuroscience does not know enough about biological processing architectures to judge what is the basic central cognitive code, although there is a broad consensus that it is non-verbal. Nor, indeed, is the code open to introspection because our output sentences (written or spoken) come to us out of the inner darkness more or less ready to deliver [this point from Velmans (1991), although we also strongly recommend Jeffrey Gray on this subject]. We shall be returning to this point in Part 7. >>

�

Masterman is also important because she suggested a major improvement to the Richens-Booth continuous form interlingua. In Masterman (1957), she simply took her copy of Roget's Thesaurus from the shelf, and used its headings as a ready-made semantic interlingua, complete with associated numeric codes. We have ourselves discussed this approach in our paper, "The magical name Miller, plus or minus the umlaut" (Smith, 1997b), where we gave some of the credit for the idea to the 19th century professor of linguistics, Max M�ller .....

�

"[M�ller] was one of the first to recognize that word meanings are constantly evolving, and spent a lifetime studying the etymologies of words in a variety of languages with a view to tracking down their derivations (M�ller, 1887). Working in this way, he reduced all languages to much smaller pools of word roots and fundamental concepts. To give but one example, the Greek words for lift, compare, tribute, spread, delay, bury, madness, endure, recover, reproach, help, and excel (to name but a few) all derive in various ways from f e r , 'to bear'. M�ller's central argument goes as follows: 'Give us about 800 roots, and we can explain the largest dictionary; give us about 121 concepts, and we can account for the 800 roots. Even these 121 concepts might be reduced to a much smaller number, if we cared to do so' (op. cit., p551.). But not all authorities agree on where to draw the line. Whereas M�ller chose a pool of 121 'original concepts', Saban (1993) chooses 31 basic 'semantic fields', and in the original 1852 edition of his renowned thesaurus Peter Roget went for 1,000 'related ideas'. So why the disagreement? Well insofar as it was responsible for importing the concept of the bit from engineering into psychology, the answer lies in George Miller's paper [ie. the Miller (1956) "Magical Number Seven" paper - Ed.]. This is because the logarithmic nature of the bit renders 31, 121, and 1,000 not so different after all: to be precise, they might just represent three points on the powers of two dimension. That is to say, we need five bits of information to specify a single one of Saban's 31 semantic fields, seven bits of information to specify a single one of M�ller's 121 original concepts, and ten bits of information to specify a single one of Roget's 1000 related ideas [raising two to the powers five, seven, and ten gives 32, 128, and 1024 respectively - Ed.]. Could we, therefore, be looking at a semantic memory which somehow operates according to a binary chop principle?" (Smith, 1997b, pp204-205)

�

NB (14th July 2003): Although ignorant of her work at the time, we owe it to Masterman to point out that she was there 40 years before we were. This from Masterman (1957, p41): "..... in any kind of writing which builds up into an argument, thesaurus-heads tend to be introduced in powers of two, each topic being introduced concurrently with that to which it primarily contrasts". It is simply a very efficient way to think about things - as Hamlet found out with all his to being and not to being. Nor, it turns out, was M�ller really the first, either, for he was simply following suggestions from the 17th century "universal languages" of Cave Beck and Athanasius Kircher. We might also add that Richens' 80 or so "naked ideas" and Parker-Rhodes' (1978) 150 or so "primitive items" are not that far from the line.

�

So to put things back into perspective, what the early MT researchers had done was to re-open a very old (and more than normally wriggly) can of philosophical worms, and in retrospect it is not really surprising that Booth and Richens did not keep their MT lead (such as it was) for long. The ARC was little more than a hobby kit computer, and the big machines soon overtook it, and - the pessimists apart - there was still a common presumption that things would be better when newer, faster, machines with more powerful programming languages came along. Nevertheless, what the Birkbeck (Booth), MIT (Yngve, Bar-Hillel), Seattle (Reifler), and Cambridge (Richens) groups had been doing was laying the foundations for the modern science of "computational linguistics".

�

4.2 Semantic Networks

Sadly, MT's problems did not go away, even with interlinguas and inflection markers and more powerful computers. Irony, ellipsis, context, and idiom, still had to be overcome. Reifler himself gives the example of trying to translate "he is an ass" into Chinese, assuring us that you cannot simply substitute the Chinese word for ass because Chinese folklore does not regard the ass as inherently stupid. There are other traditionally difficult and highly illustrative problem phrases, perhaps the most famous of which is "out of sight, out of mind", which in the absence of some extremely nifty programming invites translation as "invisible lunatic" - perfectly logical in its own way, but also perfectly stupid. Another commonly cited howler comes from the English-to-Russian MT software which reputedly rendered "The spirit is willing, but the flesh is weak" as "The steak's off, but the vodka's OK". Warren Weaver himself summarised all the criticisms in a privately (but widely) circulated memorandum on the subject (Weaver, 1949), and it was this which drew broader academic attention to the underlying issues. Thus it was that linguists and philosophers such as Bar-Hillel grew deeply pessimistic about MT's long term prospects [wait and see what he says about John McCarthy in Section 4.7!]. You could not do MT, in Bar-Hillel's view, until you had acquired a far better understanding of linguistics, and those who already understood linguistics were already wagering that you would not be able to do it then either (and, as we shall be seeing in Part 5, nobody half a century later has come close to relieving them of their money).

�

Yet the CLRU still had the fourth of its four main themes - lattice theory - in reserve. The main worker here was a colleague of Masterman, Arthur Frederick Parker-Rhodes, and his basic proposal was that the mathematics of networks (lattice theory arose a branch of Boolean algebra) was also the mathematics of language [example] .....

�

BIOGRAPHICAL ASIDE - ARTHUR FREDERICK PARKER-RHODES: text to follow

�

Drawing on Parker-Rhodes' ideas, Masterman re-analysed the linear array of meanings laid out in Roget's Thesaurus as the nodes of a large knowledge network.

�

Of course, the idea that the mind might organise its available knowledge as a vast interlocking network of concepts is not new. One of today's leading experts on the technical structuring of knowledge, ex-IBM man and knowledge consultant John F.�Sowa, for example, dates the idea to the Greek philosopher Aristotle. Aristotle's early thoughts on the hierarchical nature of definition survived in the network diagram drawn up by Porphyry in the third century CE, a device now known as the "Tree of Porphyry" [picture]. The idea was then reborn in the "Associationist" tradition of Western philosophy (led by David Hartley in the mid-18th century), and was also used by Jung as the basis of his "association of ideas" projective test [the famous "tell me the first word that comes into your head" test]. We shall be returning to this issue in Part 5, but mention it here because Richens - geneticist, self-taught database designer, and now linguistic philosopher - has recently been identified as the man who coined the term "semantic net" [source], and semantic networks are a very important topic .....

�

Key Innovation - Semantic Networks: A semantic network is "a graphic notation for representing knowledge in patterns of interconnected nodes and arcs [..... and] what is common to all semantic networks is a declarative graphic representation that can be used either to represent knowledge or to support automated systems for reasoning about knowledge" Sowa (2003 online). As we shall be seeing in Part 6, semantic networks exist within the computer world in the form of the network-structured (aka "codasyl") database. << AUTHOR'S NOTE: It is worth noting at this juncture (a) that the concept of the semantic network has made some impact on the psycholinguistic side of cognitive science, but (b) that research has been fragmented between linguistics, psycholinguistics, and mainstream cognitive theory, and (c) that one major perspective has been strangely overlooked, namely the portfolio of diagramming and analysis tools used to build data networks for use in computer databases. The most ambitious attempt so far had been by Cycorp's Douglas B. Lenat, whose Cyc project has been under development since 1984, but more on that in Part 5. Our own preference is for the "Bachman Diagram" [specimen] approach, but there is nevertheless considerable explanatory value in both Anderson's and Lenat's approach. For his part, Sowa explicitly equates his Definitional Network with Tulving's (1972) "semantic" type of LTM, and his Associational Network with the "episodic" type. However, to the best of our knowledge, there has not been a single attempt to use codasyl-type database in AI simulations. The full evaluation will not be available until 2004. >>

�

The main psychological input to the semantic network debate has been from Anderson (1983) and his "propositional network", as reproduced at Figure 2(a). However, very similar diagrams are heavily used within the IT industry to formalise the layout of large computer databases, and a specimen of one of these is reproduced at Figure 2(b). Curiously enough, attempts to import database concepts into psychological explanation are few and far between.

�

Figure 2 - Propositional Networks, (a) Biological and (b) Computer: In diagram (a) we see how a simple network diagram can be used to analyse something as philosophically complex as a unit of truth. �As is common with this sort of diagram, the individual concepts are shown as concept nodes in the network, whilst the real knowledge exists in the form of propositional nodes (the oval ones). The propositional nodes are vital because they "encode" facts (ie. propositional truths) about the concepts they link. The oval symbol on this occasion serves to link the otherwise separate conceptual nodes <LAWYER> and <PARK>, and in so doing it identifies a number of essential linguistic properties as shown in italics on the line linkages. Thus <LAWYER> is the subject of the proposition, <PARK> is the location, and <IN> specifies the relation between the two. In diagram (b), we see similar principles at work in a computer data model. This particular modelling convention uses four types of symbol. The rectangles and diamonds denote different types of concept, or entity, nodes, the ovals denote the attributes, or properties, of those entities, and the lines denote the relationships between entities and other entities or between entities and attributes. Again, the essence of the relationships between entities is marked on the line linkages. Properly implemented by a Database Management System, every occurrence of every entity type, attribute, and relationship can be safely stored in rapid access computer filestore, and retrieved in short order. This is precisely the sort of technology which would have allowed Reifler [see Section 4.1] to squeeze his 15 drum dictionaries onto a single storage resource.

<![if !vml]><![endif]> � Copyright � 2003, Derek J. Smith. Previously in Smith (1996). Redrawn from black-and-white originals (a) in Anderson (1983, p26), and (b) in Oxborrow (1989, p36).

�

For a rich selection of other diagrams and approaches, see Sowa (2003 online). In fact, Sowa (2003 online) identifies no less than six subtypes of network, as follows .....

�

(1) Definitional Networks: These are networks which define a concept in terms of its similarity to other concepts. Sowa calls this a "subtype", or an "is-a" relation, and places its value in the fact that the properties of the supertype are automatically "inherited" by the subtype. [Our Example: <ROBIN> is a <BIRD, SMALL, GARDEN, NON-MIGRATORY>.]

�

(2) Assertional Networks: These are networks which assert propositional truths. [Our Example: <DAVID> <IS EATING> <CAKE> or <WHEN HE WAS 7> <DAVID> <VISITED> <SEATTLE>.]

�

(3) Implicational Networks: These are networks which "use implication as the primary relation for connecting nodes". [Our Example: <THE ROCK IS AT THE TOP OF A SLOPE> which implies ,..... AND SO IS LIKELY TO ROLL IF PUSHED>.]

�

(4) Executable Networks: These are networks which have mechanisms for actively locating content according to a prescribed search mask. [Our Example: A network which could respond appropriately to an instruction in the form LIST ALL CONTENT ASSOCIATED WITH THE TARGET <SLOPE>.]�

�

(5) Learning Networks: These are networks which can extend themselves, either by adding new nodes or new arcs between existing nodes, or by modifying the association strengths, or "weights", of arcs. [Our Example: A face recognition network which can learn to recognise new faces (new nodes), or attach new attributes to existing ones (new arcs).]

�

(6) Hybrid Networks: These are networks which combine two or more of the preceding types.�

�

4.3 Mechanised Game Playing

In 1769, the inventor-showman Wolfgang von Kempelen built a chess-playing automaton which he demonstrated at theatres and fairs where it proved capable of beating most contestants. Unfortunately, von Kempelen was pulling a fast one, and the functional CPU turned out to be the world's first minicomputer - a human dwarf secreted within the device [picture and fuller story]. The idea remained appealing, however, and one of the pastimes of the first computer programmers was to investigate whether computers could be programmed to play games such as draughts [US = checkers], noughts and crosses [US = tic-tac-toe], or chess. The pioneer thinker in this area seems to have been Germany's Konrad Zuse [reminder], inventor of Plankalk�l, the world's first higher-level programming language. In the period 1942-1945, Zuse sketched out designs for a chess playing program for his Z4 computer, but it never got written and the design was only published in 1972 [source]. The formal glory goes instead to another of Warren Weaver's proteg�s, Claude Shannon of the Bell Telephone Company. In a manuscript dated October 1948, Shannon analysed the problems which were likely to be encountered when getting a computer to play chess (subsequently published as Shannon, 1950/1993). He saw this as one of many ways in which computers could move in the direction of processing "general principles, something of the nature of judgement, and considerable trial and error, rather than a strict, unalterable computing process" (Shannon, 1950/1993, p637). He saw "the chess machine" as an ideal problem with which to start the quest for what he called "mechanised thinking", and discussed methods of codifying and then evaluating proposed moves. Not surprisingly, Shannon's papers inspired some ground-breaking computer programming in the early 1950s. The following bullets come from a number of sources, including Copeland (2000 online) .....

�

• The Bletchley Park veteran, Donald Davies, published an article on the mechanisation of board games (Davies, 1950), and coded a simple noughts-and-crosses program for the NPL's Pilot ACE.

�

• In May 1951, the aforementioned Christopher Strachey [see Section 2.3], wartime radar boffin turned schoolteacher, scrounged time as a technical hobbyist on the NPL's Pilot ACE, and wrote what has been acclaimed as the world's first AI program - a draughts program. Unfortunately, the code had a few bugs in it, and Strachey did not get it working tidily until the following summer on Manchester's Ferranti Mark 1 (Copeland 2000/2003 online).

�

• In November 1951, Manchester University's Dietrich Prinz used Turing's newly produced "Programmer's Handbook" to produce the first experimental chess program, also on the Ferranti Mark I (Copeland, 2000 online).

�

• IBM's Arthur Samuels heard of Strachey's program at a 1952 conference, and adapted it for the IBM 701. He then had to spend the next ten years improving it, eventually adding the rudimentary ability to learn from experience, whereupon the code turned rather churlishly on its creator and started to beat him (Samuels, 1959).

�

• IBM's Alex Bernstein wrote the first reasonably competent chess program on an IBM 704 in 1957.

�

• With even more improvements, Samuels' 1959 program beat a human draughts champion in 1962. The program is historically significant because it used "heuristics" rather than "algorithms" to evaluate and score possible moves.

�

Key Innovation - Algorithms vs Heuristics: The word "algorithm" derives from the Arabic "al Khowarasmi", the popular name of the ninth century Arab mathematician Abu Jafar Mohammed Ben Musa, and refers to a sequence of formal actions by which a given problem can be systematically resolved. It is "a systematic list of instructions for accomplishing some task" (Wikipedia, 2003). Thus computer programs are algorithms. The word "heuristic" derives from the Greek root euriskein - "to find" (thus being related to "Eureka" = "found it!"), and, strictly speaking, refers to anything which promotes an act of discovery. However, where algorithms consist of specific actions, heuristics consist of rule-action combinations, so with an heuristic you do not know in advance what you are going to do - you have to wait and see what circumstances trigger what rules. The beauty of the if-then approach is that it can save you a lot of time. When playing chess, for example, it is rarely a good idea to exchange a queen for a pawn, and so an entire branch of the exhaustive search tree can be discounted at a stroke. Insofar as the word is used in game theory, the point about heuristics is that they attempt rapid penetration of a problem rather than exhaustive analysis, by virtue of which they are often referred to as "rules of thumb". Ultimately, of course, both algorithms and heuristics are only as good as their designers. If either is not particularly powerful, then the quality of the penetration will be limited.

�

4.4 Man-Machine Conversation

The fourth line of enquiry was a more philosophical one, and looked at what needed to be done to turn a cleverly programmed calculator into something qualitatively more than a cleverly programmed calculator. What, for example, might a cash register have to do to be accepted as containing a mind; something more than just a thing which can be switched on and off on demand? What might your laptop have to do to be accepted as possessed of some kind of vital spark? This question was first put forward in a paper from Mr GPC himself, Alan Turing, who, when not codebreaking, was "really quite obsessed with knowing how the human brain worked and the possible correspondence with what he was doing on computers" (Newman, 1994/2003 online, p12). Taking time out from his work on the Manchester Mark 1, Turing wrote a paper entitled "Computing machinery and intelligence" in which he suggested that the question "can machines think?" was philosophically unsafe, due to problems agreeing the meaning of the word think (Turing, 1950/2003 online).

�

Of course, Turing had long believed that all but the most obscure of problems could be solved by a sequence of simple actions, and already for a decade programmers had been proving him right. He now recommended that this sequence of actions be thought of as having an existence and meaning in itself: a program, once devised, was a solution in prospect, as it were, and simply "awaited" a machine to execute it. Turing then brought these two propositions together by arguing that the mind would not only prove one day to be "programmable", but that the eventual program would be "implementable" on a machine. And once you could do that, he said, you would have a machine which would be indistinguishable from a human; that is to say, a machine which could think. As to how you would test this indistinguishability, he proposed objectively establishing whether the machine, so programmed, could perform as successfully as a human in fooling an interrogator in an "imitation game", in which a man (A) and a woman (B) have to fool (C) as to which is the man and which is the woman. The knowledge pertaining to (A) and (B) is accumulated in the mind of (C) by asking questions, the only restriction being that the answers to those questions should be typewritten so as to prevent vocal clues being given.

�

Turing's imitation game evolved somewhat over the years, and in its later form became popularly known as the "Turing Test". This runs as follows: if a human in room A were to communicate via keyboard and screen with an entity in room B which might be a human but which might also be a computer trying to appear human, then the definition of "humanness" would rest on whether the real human could tell the difference or not after five minutes of questioning. Serious academic attempts to "pass the Turing Test" began in the 1960s, and we shall be looking at some of these in Part 5. Reader (1969) even sketches out the processing modules which would be necessary to build such a machine [check it out], but the layout he proposes turns out to be yet another instance of the cognitive control hierarchy, indistinguishable in essence from Lichtheim's (1885) "House" diagram, and surpassed in supporting detail by Frank's (1963) "Organogramm".

�

4.5 "Neurodal" Networks and Machine Learning

The fifth line of enquiry was pitched more at the hardware side of things, and can be traced to a number of seminal papers from the 1940s, led by the American neuropsychologist Warren S. McCulloch (1898-1968). The first of these was McCulloch and Pitts (1943), an attempt at mathematically modelling the workings of biological neurons. The authors called their artificial neurons "neurodes", and each was a simple arrangement of electronic components designed to do two main things, namely (a) to output a signal similar to the bursts of action potentials output by biological neurons, and (b) to ensure that this output was modulatable - sensitive to what other neurodes in the vicinity were doing. In this way, quite simple combinations of neurodes were capable of performing the logical operations AND, OR, and NOT to much the same end effect as the electronic circuitry then being developed for use in computers. Pitts and McCulloch (1947) then went an important step further, by discussing what would happen if you took many such neurodes and wired them together. Specifically, they proposed that it would be possible to end up with some sort of artificial pattern recognition system. And McCulloch and Pfeiffer (1949) considered how binary principles might exist within biology. They portrayed the nervous system "as if it were a wiring diagram and the physiology of the neuron as if it were a component relay of a computing machine" (p369). Their aim was then to "set up working hypotheses as to the circuit action of the central nervous system as a guide to further investigation of the function of its parts, and as a scheme for understanding how we know, think and behave" [ibid.]. This is part of their observations .....

�

"Judging by the speed of response, neurons fall into the middle range of man-made relays. They are about a thousandth as fast as vacuum tubes, about as fast as thyrotrons, but faster than electromechanical and mechanical devices. Because they are much smaller than electronic valves, though the voltage gradients are about the same, they take lower voltages and less energy. A computer with as many vacuum tubes as a man has neurons in his head would need the Pentagon to house it, Niagara's power to run it, and Niagara's water to cool it. The largest existing calculating machine has more relays than an ant but fewer than a flatworm. Neurons, cheap and plentiful, are also multigridded, the largest having upon them thousands of terminations, so that they resemble transistors in which the configuration of electrodes determines the gating of signals." (McCulloch and Pfeiffer, 1949, p370)

�

HISTORICAL ASIDE: In fact, the idea of nerve net computation is far older than the attempts to construct it. One of the first to suggest that learning was accompanied by the formation of new synaptic connections was the Aberdeen logician, Alexander Bain (1818-1903). Inspired by the work of Lionel S. Beale (1828-1906), pioneer in medical microscopy, and specifically by Beale (1864), Bain tried to find a likely neuroanatomical substrate for the sort of semantic networks popular with psychologists of the "Associationist" persuasion. The process relied on the "contiguity" - that is to say, closeness together in time or space - of the elements to be associated. "Contiguity," he wrote, "joins together things that occur together, or that are, by any circumstance, presented to the mind at the same time ....." (Bain, 1879, p127). He set out his technical proposals in his 1873 book "Mind and Body", from which the following clearly indicates his line of argument: "For every act of memory [.....] there is a specific grouping, or co-ordination, of sensations and movements, by virtue of specific growths in cell junctions" (Bain, 1873, p91; cited in Wilkes and Wade, 1997). Bain then analysed differently interconnected clusters of neurons using simple abstract circuit diagrams (reviewed by Wilkes and Wade (1997). The next major historical figure was Santiago Ramon y Cajal (1852-1934) (eg. Ramon y Cajal, 1911), whose general idea was that plasticity in neuron-to-neuron connections - what are now known as "synapses" - made it possible for neural pathways to connect themselves up on demand - that is to say, as learning took place. Pathways could thus appear where previously just disconnected neurons had existed. Moreover, the biological act of creating those pathways underpinned the psychological act of memorising the experience in question. Some years later, Rafael Lorente de No (1902-1990) extended the debate by microscopically analysing the layout of the synaptic "buttons" on neural cell bodies (Lorente de No, 1938). Working at the limits of magnification of the microscopes then available, he found many hundreds of synaptic buttons dotted about the neural membrane, thus reinforcing suspicions as to their role in neural circuitry. Subsequent studies have confirmed this view, and with today's very powerful microscopes veritable forests of synaptic buttons can be seen, and not just on the neural soma but out on the dendritic trees as well. Another of Lorente de No's contributions was to show how interneurons could be used to feed excitation back into a given neural circuit, thus keeping that circuit active long after the original source of excitation had been removed. This arrangement is commonly referred to as a reverberating circuit. However, the most influential exposition of neuronal net theory came from Donald O. Hebb (1904-1985). In his 1949 book, "The Organisation of Behaviour", Hebb described the interlinking of neurons as creating what he called a cell assembly, which he described thus: "..... a diffuse structure comprising cells in the cortex and diencephalon (and also, perhaps, in the basal ganglia of the cerebrum), capable of acting briefly as a closed system." (Hebb, 1949, p. xix). Hebb's ideas of how cell assemblies form and function can be gathered from the following: "The general idea is an old one, that any two cells or systems of cells that are repeatedly active at the same time will tend to become 'associated', so that activity in one facilitates activity in the other" (Ibid, p70). The idea of cells repeatedly assisting each other's firing is particularly well brought out in what has since come to be known as "Hebb's Rule".

�

The next step was to go ahead and wire a few neurodes together, and in 1951 Marvin Minsky did just that (Minsky, 1994). As part of his postgraduate researches in the mathematics department at Princeton, and in conjunction with Dean Edmonds, he built SNARC (= "Stochastic Neural-Analog Reinforcement Computer"), the first machine to display learning behaviour, thanks to an inbuilt reinforcement-sensitive 40-neuron "neural net". This was followed a year later by William Ross Ashby (1903 - 1972)'s "Design for a Brain" (Ashby, 1952). Ashby was the cyberneticist who had popularised the concept of "homeostasis" in biological systems. He therefore took a more macroscopic view of neural architecture than had McCulloch or Minsky, that is to say, he was more concerned with what the nervous system was trying to achieve as an organic whole with interrelating modules, rather than with what individual neurons were up to. His work with biological homeostasis had convinced him that many biological control phenomena were simple negative feedback systems, save that they had biological sensors and comparators, and during the 1940s he had experimented with a simple machine called "the homeostat", which was sensitive to environmental events and could initiate corrective behaviurs to neutralise any "perturbation" in their stable settings.�Angyan (1959) reports on a more advanced machine called Machina Reproducatrix, fitted with enough sensors and motors to give it a light-seeking "tropism", however we need to talk about this in Part 7, and so will leave the details until then.

�

ASIDE: The term "tropism"was coined by Loeb (1888, 1890) to describe the wholly involuntary behaviour of organisms lacking a nervous system. Just as plants can turn towards the sun using non-nervous mechanisms, so, too, is much simple animal behaviour unwilled. To be classified as a tropism, the relationship between the stimulus and the response must be rigid and resulting directly from the action of some external energy source. Tropisms can be subcategorised by energy type and direction of response as follows: Geotropisms: positive - turn towards the pull of gravity; negative - turn away from the pull of gravity, Heliotropisms (or phototropisms): positive: turn towards the light; negative - turn away from the light, Thermotropisms: positive: turn towards heat; negative - turn away from heat, and Rheotropisms: positive: turn upstream (in air or water); negative - turn downstream. In animals with nervous systems, this type of behaviour is provided by reflexes. For a longer introduction to the various types of innate and learned animal behaviour, see Section 1 of our e-paper "Communication and the Naked Ape".

�

In fact, 1952 was a busy year, because it also saw Claude Shannon following up his chess machine with the blueprints for a maze-solving mechanical mouse (Shannon, 1952/1993), and Anthony G. Oettinger writing a program called "Shopper", the first program (as opposed to learning hardware) capable of learning from experience (Oettinger, 1952). Copeland (2000/2003 online) describes Shopper as operating in a virtual world containing a mall of eight shops stocking a number of items. Shopper would be sent off to purchase a particular item, and would have to visit shops at random until successful. The learning lay in the fact that Shopper would memorise some of the items stocked in the unsuccessful visits. Subsequent shopping expeditions, either (a) for repeat purchases of the first item, or (b) for first time purchases of one of the memorised items, would be right first time, thus showing the value of short term memory resources in supporting adaptive behaviour.

�

We must also mention Albert M. Uttley, whom we first met in Section 2.2 as the TRE boffin who designed TREAC, the first British parallel processing computer. Over the years, Uttley gradually extended his interests to include biological as well as workbench electronics. Like McCulloch, he was particularly fascinated by the logic by which neurons were interconnected, and was one of the first to describe the sort of statistical connection rules which might control the biological learning process. In Uttley (1955), he derived mathematical formulae to describe such things as dendrite distribution and axonal connections.

�

However, the most famous of the early machines was built by Frank Rosenblatt in the mid-1950s, and called a "perceptron". The essence of Rosenblatt's design was that the neurodes were arranged into two banks, or "layers". One of these was connected to an artificial eye and called an input layer, and the other was connected to the chosen output mechanism (usually an array of flashing lights) and called an output layer. Each point in the input layer was wired to each point in the output layer, and the effective strength of these connections was specifically designed to be varied. Learning was then a process of changing the strength of the connections - a process known as weighting. As we shall be seeing in Part 5, McCulloch's neurodes and Rosenblatt's perceptron formed the basis for one of modern science's most active and exciting research areas, namely "Connectionism".

�

The point about perceptrons is that the weighting process delivered learning by experience. There was no pre-written program telling the hardware what to do - rather the machine worked it all out for itself. Perceptrons took the academic world by storm, therefore, especially where there were problems which were proving beyond the programmers in the first place - such as most of higher cognition. Thus .....

�

"It is not easy to say precisely what is meant by learning or what corresponds to it in a machine. [.....] The machine is able to learn up to a point, but once that point has been passed it is helpless. Here it differs from a human being, who is able to learn and to go on learning. [//] A program which would enable a machine not only to learn but also to go on learning, and to adapt itself to each new situation as it arose, I will call a 'generalised' learning program. If such a program could be constructed it would presumably consist of a relatively small initial nucleus of orders, and would adapt and extend itself as circumstances required, replacing its subroutines and switching sequences by others of a more general kind, and creating new subroutines to perform fresh functions. There does not appear to be any reason for believing that the construction of such a program is inherently impossible [.....] Although limitations of speed and storage capacity would undoubtedly soon make themselves felt, it is not these so much as shortage of ideas which is at present holding up progress." (Wilkes, 1956, pp291-293; emphasis added)

�

4.6 Robotics

The defining essence of a robot is that it carries out work on behalf of its master. Strictly speaking, therefore, a robot needs bear no physical resemblance to any biological form (although, for reasons which are still being looked into, we seem to prefer that they do). As such, the broad concept can be traced back several thousand years [see our e-resource on the history of automata]. Even in 1950, the name "robot" was already at least 30 years old, being usually credited to Karel Capek's 1921 play "Rossum's Universal Robots", and Isaac Asimov's "Laws of Robotics" were 10 years old. As for the technology itself, it was at least 40 years old, and had been forged, as always, on a war anvil .....

�

HISTORICAL ASIDE - MILITARY ROBOTICS: The military have always proved generous sponsors for companies specialising in technological innovation, and we have already described elsewhere [reminder] how the Sperry Gyroscope Company grew into a major international corporation on the back of mechanical and electromechanical naval control systems. Sperry were also quick to see the same opportunity for fully automated aerial weapon systems, and as early as 1913 were using gyroscopic stabilisers as part of a radio-controlled aircraft (Pearson, 2003 online). The demands of the First World War extended the science of military robotics into the control of aerial torpedoes and the design of better bombsights and perhaps the best known products of this early research were the World War Two German V-1 and V-2 pilotless rocket systems, which both had autopilots and onboard guidance computers capable of steering them several hundred miles.�Much less spectacular, but deadly nonetheless, were the Ruhrstahl SD1400 "Fritz X" and the Henschel Hs293A "stand-off" bombs [picture; pictures and discussion]. These were "Lenk-", that is to say guided, "bomben", and were both small winged glider bombs designed for anti-shipping use. The SD1400 was heavy and free-falling, with an armour-piercing capability, whilst the Hs293 was smaller and rocket-assisted, but lacked the armour-piercing capability. They were released from their mother aircraft while still a number of miles from the target, and steered on their short one-way flight by (initially) radio signals generated by a small joystick [picture]. The Lenkbomben were developed in the period 1940-1943, and when ready for field deployment were issued to specially equipped Luftwaffe squadrons codenamed KG40 and KG100. History's first successful guided missile attack on a surface ship then took place on 27th August 1943 in the Bay of Biscay, when the corvette HMS Egret was sunk and the Canadian destroyer Athabascan damaged by Hs293s launched from KG100 aircraft. The following month, the cruisers HMS Uganda and USS Savannah, together with the battleship HMS Warspite, were seriously damaged by Fritz Xs while supporting the Salerno landings, and the Italian battleship Roma was sunk by one or two hits [accounts differ] from Fritz Xs while attempting to defect to the Allies under the terms of the Italian surrender. The battleship Italia was also hit on this occasion, and her escort, the battleship HMS Valiant, damaged by a near miss, but both managed to make their way to Malta [source]. In November 1943, the British troopship Rohna was also sunk en route from Oran to Bombay carrying American troops, and this incident, in terms of lives lost (1138 dead), was actually a worse disaster than the attack on Pearl Harbour (Cassanello, 2003 online) [fuller story; in memoriam]. Still in the Mediterranean, the cruiser HMS Spartan was then sunk off the Anzio beachhead in January 1944 [fuller account]. The Allies responded very quickly to this new age threat, analysing the German radio control frequencies, and adding radio countermeasure systems (="RCM", or "jamming" equipment) to their front-line naval assets as fast as the equipment could be built and installed. This upgrade extended to a number of landing ships being fitted out as Fighter Direction Tenders (FDTs) in readiness for the 1944 invasion. These vessels were selected because they would be stationed off the landing beaches in a command and coordination role alongside existing headquarters ships such as HMS Largs, Hilary, and Bulolo. As a result, when the invasion went ahead in June 1944 the Allied fleet emerged relatively unscathed. For example, we find the following entry in the log of the battleship USS Texas: "the day [7th June] started with a detection of several anti-ship missile radio signals and subsequent jamming" [source]. KG100 was active for a few days off the Normandy beaches, and scored at least one sinking, that of HMS Lawford on 8th June [divers examined the wreck in 2002, and the nature of the damage confirms that a glider bomb was the most likely culprit]. Off Arromanches, they also hit, but did not sink, HMS Bulolo [picture], the Force G (Gold Beach) headquarters ship, striking her - fortunately for the ship as a whole, if not those actually up there - high on the superstructure [the specialist German bomb-aimers were trained to aim at their target's waterline for maximum effect, but even under ideal conditions were lucky to hit within a five metre radius]. The Germans were apparently slow to respond to Allied RCM with "counter-countermeasures", but eventually replaced the radio controlled Hs293A with the wire-guided, and therefore RCM-immune, Hs293B. However, as the war moved progressively inland the opportunity to field this improved system was lost, and the specialist squadrons were disbanded. In all, 319 Fritz X and Hs293 missiles are reported to have been launched in action, of which roughly 100 scored hits. 40 warships and 30 merchant ships were damaged or sunk [source]. In addition to the ships named above, German war historians also claim the destroyers HMS Jervis, Inglefield, Boadicea, and Intrepid as sunk, and the cruiser USS Philadelphia damaged [confirmation]. In addition, one of the aforementioned FDTs, FDT 216, is on official record as having been sunk by an air-launched "torpedo" on 7th July 1944, but it is unclear from the sources available to us whether this was of the guided or free-running type. The next major advance with tactical rocketry was the "homing" missile. These were developed typically for use against either aircraft or submarines, and the functional principle is that the missile "locks on" to an external energy source (typically the engine exhaust heat of an aircraft or the propeller noise of a submarine) and then twists and turns as much as is necessary to keep that source in its sights (allowing for deflection as necessary). This type of system reached a state of some sophistication in air-to-air missiles such as Sidewinder. Laser guided bombs were developed during the 1980s and saw action during the 1991 Iraq War. They have since been partly replaced by JDAM smart bombs [News24 picture

], where the projectile homes on coordinates programmed into a highly flexible tactical command system, and conveyed to the weapon after its release by signals from a satellite. JDAMs are a few metres less accurate than laser-guided munitions, but are cheaper, can be used in larger numbers at a time, are "fire and forget", and are not affected by cloud, darkness, or smoke. [For a fuller introduction to this type of weapon, click here, for more on the technicalities see Blackwelder (1992/2003 online), for an overview of modern military robotics, click here, and for an introduction to how modern "smart" bombs work, click here.]

�

So we now have something of a pattern emerging. With conventional munitions, the projectiles are totally "dumb" - mere lumps of iron, devoid of any form of intelligence; no more robots than spears or musket balls are robots. With the operator-guided weapons, the projectiles are responsive, but still dumb - they make no decisions of their own, but instead receive input (from the remote joystick) and generate output (to their steering mechanism). And with the early pilotless rocket systems, all the control systems were pre-set before launch, and the missile followed the programmed trajectory as closely as it was physically able. This mixture of technologies means that in the typical air-ground combat engagement of modern warfare we often pitch significantly different control systems against each other. We might, for example, have a tactical dual between a ground unit armed with a fire and forget shoulder-launched surface-to-air missile such as a Stinger, and a helicopter armed with a laser-spotted air-to-ground round such as Hellfire [pictures and details], or between a city's SAM batteries (radar-guided, passive or active), and a B-2 loaded with JDAMs. The common denominator, of course, is that all forms are ultimately placed on target by a human, even if the act of aiming took place at a keyboard many thousands of miles away and many weeks beforehand. The intelligence (and we use the word cautiously) is in the mind of the operator whose eye is on the target (literally or figuratively) and whose hand is on the joystick. So the machines are not really intelligent at all. Even the JDAM has no "flexible control over the sequence of its operations" (Berkeley, 1949, p5), nor is it likely to acquire any for a good few years yet. In fact, Carnegie Mellon University's Hans Moravec [more on whom in Part 5] doubts that such "hazardous duty" systems should be called robots at all - he prefers the title "autonomous machines", because "they have the bodies of robots, but not the brains" [source].

�

ASIDE: We use the word "intelligence" cautiously, because it is one of those words which can mean whatever the speaker at the time wants it to mean. It can mean advanced philosophy to some, contemplative planning to others, or just the ability to hold a target in the crosshairs, and it is precisely this vagueness which has kept the Turing Test [see Section 4.4] so relevant all these years. The focus of our attention must therefore turn away from robotic mechanisms and design architectures, and look instead directly at the intelligence we are trying to inject into our robotic brain .....

�

4.7 Machine Problem Solving

We turn, therefore, to the highest of all forms of guidance, namely reasoned action, and the nearest AI got to this in the 1950s was in its experiments with machine problem solving. The most successful of these was the collaboration between the Rand Corporation's Allen Newell and J. Clifford Shaw, and Carnegie Mellon University's Herbert A. Simon (1916-2001) to investigate how machines might profitably mimic (and potentially help explain) human mathematical problem solving. One of their programs is still regarded as defining the science .....

�

The "Logic Theorist": This 1956 program was designed to test the truth of specific mathematical theorems. It did this by allowing the programmer to input five basic propositions (called "axioms"), against which were applied a number of "rules of inference". This enabled further logical propositions to be derived, by a process based upon the human process of formal mathematical argument, and as each derived sentence is generated it can be checked to see if it matches the theorem under test. If it does, then the audit trail of successive inferences by which it was derived automatically constitutes the mathematical proof required. If it does not, then the program advances to the next permutation. Unfortunately, the problem with this approach is basically one of time, because ideally you would have to explore every combination of axiom and rule through every possible iteration (a brute force computational approach sometimes known as the "British Museum algorithm") [formal definition]. Remembering the distinction we raised in Section 4.3, this is therefore an excellent example of problem solving by algorithm rather than by heuristic.

�

Logic Theorist was the central exhibit in 1956 at a conference on AI organised at Dartmouth College by John McCarthy (1927-), one of their mathematics lecturers .....

�

BIOGRAPHICAL ASIDE - JOHN McCARTHY: McCarthy had been following developments in machine cognition very closely, and had already coined the general descriptor "artificial intelligence" in 1955. He would go on in 1958 to develop the LISP programming language, and in 1959 wrote an influential paper entitled "Programs with Common Sense" in which he defined machine common sense as follows: "..... a program has common sense if it automatically deduces for itself a sufficiently wide class of immediate consequences of anything it is told and what it already knows." (McCarthy, 1959/2003 online). As to how this might be achieved, reference should be made to the five system features proposed in the original paper, if interested. However, the fifth of these features in particular has turned out to be remarkably prescient .....

�

"5. The system must be able to create subroutines which can be included in procedures as units. The learning of subroutines is complicated by the fact that the effect of a subroutine is not usually good or bad in itself. Therefore, the mechanism that selects subroutines should have concepts of interesting or powerful subroutine whose application may be good under suitable conditions." (Ibid., p3.)

�

McCarthy's 1959 paper is also noteworthy as an early example of the ill feeling and interprofessional disdain which can sometimes be seen between linguistics professors and the engineers who gravitate so frequently into AI. "Dr McCarthy's paper," wrote Bar-Hillel in the peer review section of the cited paper, "belongs in the Journal of Half-Baked Ideas [and] before he goes on to bake his ideas fully, it might be well to give him some advice ...." (Ibid., p11). Be that as it may, McCarthy had pointed unerringly at the central problem of any hierarchical control system, namely where to place (and how to build) the bit that makes the all-important judgements of interesting, powerful, etc. McCarthy is currently professor of computer science at Stanford University, and is well on the way to making his lifetime's work available on the Internet [index].

�

Another program, the "Geometry Theorem Prover" was reportedly inspired by a tentative program structure drawn up in 1956 by MIT's Marvin Minsky. This was a sequence of instructions capable of proving a particular geometrical problem. Minsky passed his notes to a young IBM researcher named Herbert Gelernter, who then implemented it on an IBM 704. The resulting software was capable of analysing a geometrical diagram for its constituent propositions, and of then attempting to derive further proofs in much the same way as Newell, Shaw, and Simons' Logic Theorist.

�

Newell, Shaw, and Simon struck back in 1957 with the "General Problem Solver", a program expressly designed to simulate the processes of human problem solving. Here is Copeland's (2002/2003 online) summary .....

�

"..... work continued on the project for about a decade. General Problem Solver could solve an impressive variety of puzzles, for example the 'missionaries and cannivals' problem: How are a party of three missionaries and three cannibals to cross a river in a small boat that will take no more than two at a time, without the missionaries on either bank becoming outnumbered by cannibals? GPS would search for a solution in a trial-and-error fashion, under the guidance of heuristics supplied by the programmers. One criticism of GPS, and other programs that lack learning, is that the program's 'intelligence' is entirely second-hand ....."

�

The period under consideration ended with a paper by Newell, Shaw, and Simon (1958) on what they called "the problem of complexity". In it, they surveyed the attempts to date (including their own) and the heuristics used, and concluded that the complexity of heuristic programs was beyond the power of the programming languages then available, and appealed for more powerful languages.�In Part 5, we shall be looking at what came along.

�

4.8 Artificial Consciousness

Finally, there has been a line of enquiry which has gone one important step past machine intelligence, looking for the secret of biological consciousness itself. Unfortunately, this remains an area where there is no hard data, little simulation, and a lot of speculation. As with intelligence, we simply do not know what consciousness is in philosophical terms, and so we need to be ready to come across it by accident in any of the various sub-areas of AI. It might be in the interlinguas of the verbal mind (Section 4.1), or in the data networks of the knowing mind (Section 4.2), or in the game playing heuristics of the game-playing mind (Section 4.3), or in the ability to pass the Turing Test (Section 4.4), or in some lucky artifact of the neural network (Section 4.5), or in the onboard intelligence of a third generation robot (Section 4.6), or even in the reflection which characterises good problem solving (Section 4.7). In short, we do not know what we are looking for, and that always makes life difficult for programmers [time, in fact, to call in the systems analysts ;-)]. We therefore close with quotable quotes on this very point by two of generation zero's greatest .....

�

"We know the basic active organs of the nervous system (the nerve cells). There is every reason to believe that a very large-capacity memory is associated with this system. We do most emphatically not know what type of physical entities are the basic components of the memory in question." (von Neumann, 1958, p68; emphasis original)

�

�"There is no difficulty in programming a universal machine to perform any set of operations when once the rules for performing them have been stated explicitly; this is true however complex the rules may be and however many alternative procedures of a conditional type are specified" (Wilkes, 1956, p291; emphasis added.).

�

5 - Still to Come

That concludes our review of the history of first generation computing technology during the period 1951 to 1958. In Part 5, we bring the review up to the present day, and identify several more key technical innovations, not least Database Management Systems, in Part 6 we look at computer memory in greater detail, especially as used in the COBOL programming language, the IDMS DBMS, and the TPMS TP monitor, and in Part 7 we undertake a comparative review of how well the full richness of computer memory subtypes had been incorporated into psychological theory.

�

6 - References

See Main Menu File.

[Up][Previous][Next][Home]

�