On the short-term feasibility of whole brain emulations

Ely Spears October 27, 2011

Introduction and Intractability Arguments

Whole brain emulation refers to conducting a computational simulation of the biological processes of a human brain. If we accept the premises of computationalism, then achieving consciousness is only a matter of simulation accuracy. While it may appear to be straight-forward that this is in principle achievable, it is not obvious whether the engineering barriers involved make brain emulation intractable. In [5], Ian Parberry successfully addresses some aws in the Chinese Room experiment of Searle. However, by appealing to resource constraints on the ability to emulate a brain, Parberry reaches too far and claims that even though it may in principle be possible to create sentient machines, it is intractable for humanity to do so on any reasonably short-term time scale. Parberrys argument states that the number of neurons in a human brain is about 1010 . He proposes roughly 103 connections per neuron as a conservative estimate and makes the assumption that it requires roughly one oating point operation (op) to combine two inputs into a single neuron. Because neurons re roughly once every 10 milliseconds, we end up with something on the order of 1015 ops per second of processing speed for a human brain. Modern neuroscience has left these estimates more or less alone since the publication of [5]. The number of connections per neuron is now quoted closer to 104 and the number of neurons is closer to 1011 [3], but we may leave these estimates alone: the general nature of Parberrys claim remains intact even under revised brain parameter estimates. If we acquiesce to some of Parberrys results [6] on the overhead required to simulate one Turing machine via a dierent Turing machine, then merely simulating the brains computational model on a dierent (say classical) computer will likely require a quadratic overhead in hardware or time. If this is correct then, even with rapidly advancing computational power, humanity is dramatically distant from being able to simulate the neuronal physics required for a whole brain emulation. For Parberry, this is a refutation of the possibility of strong A.I. in the foreseeable future: if evolution has made it this dicult just to simulate a human brain, then however much more dicult it is to design a brain is far outside of humanitys intellectual grasp. Recently, Microsofts Paul Allen addressed some of these issues [8], saying: [The] prior need to understand the basic science of cognition is where the singularity is near arguments fail to persuade us. ... Building the complex software that would allow the singularity to happen requires us to rst have a detailed scientic understanding of how the human brain works that we can use as an architectural guide, or else create it all de novo. This means not just knowing the physical structure of the brain, but also how the brain reacts and changes, and how billions of parallel neuron interactions can result in human consciousness and original thought. Getting this kind of comprehensive understanding of the brain is not impossible. If the singularity is going to occur on anything like Kurzweils time line, though, then we absolutely require a massive acceleration of our scientic progress in understanding every facet of the human brain. Note in particular that Allen contends that we must know how billions of parallel neuron interactions give rise to consciousness, and that one distinction between accelerating progress of, say, transistors on a chip and cognitive computation is that the latter requires dierent fundamental science and economic incentives. Like Parberry, Allen agrees that a brain 1

could in principle be simulated in a computer. Allen appears also to believe that this may potentially occur given enough time, but Allen seems to echo Parberrys claim [5] that [s]imply because cognition can be realized in the brain (in some fashion that we do not yet fully understand) with a reasonable resource requirement is no reason to believe that its resource requirements as a formal program will be reasonable too. Perhaps the main objection raised by Allen in [8] is what he calls the complexity brake. This is the notion that as more details are discovered about a system, we become increasingly aware that our current understanding is insucient to truly describe it. Allen suggests that merely knowing the anatomy of a complex physical system wont give you the ability to simulate it and that the result of most attempts [to simulate neurons] is that in order to create an adequate simulation of the real ongoing neural activity of an organism, you also need a vast amount of knowledge about the functional role that these neurons play, how their connection patterns evolve, how they are structured into groups to turn raw stimuli into information, and how neural information processing ultimately aects an organisms behavior. Without this information, it has proven impossible to construct eective computer-based simulation models.[8] Let us take stock of what has been claimed so that we may systematically address various issues. If we we want to believe that emulating a brain is tractable, we must answer several questions: (Q1) Is it feasible in an engineering sense to actually measure the characteristics of neurons and connections to such a resolution that non-trivial mental simulation could arise from them? (Q2) If we can describe such a complicated brain network state, will we have enough computational power to simulate it? (Q3) Even if we can measure the necessary characteristics of a brain network state and we have the necessary computational resources, will additional abstractions be required before we can possibly make sense of cognition from the measured brain network state? (Q4) Is there evidence of a real complexity brake preventing mind emulation? That is, does the current neuroscience of brain structure tell us that not only is a neuron-by-neuron simulation insucient to recover brain function, but also that we dont yet have any idea what would be sucient? In the sections that follow, I attempt to address these questions. First I deal with the engineering diculty of actually imaging the brain and obtaining a stored copy of a single brains network state. In the course of doing so, I oer some reasons why simply knowing this state would be sucient for simulating mental activity, provided that we have the computational resources to run the simulation. Finally, I consider a thought experiment to argue that the abstraction of Turing machine overhead is not the correct way to envision the diculty of simulating physical systems. Likewise, I discuss some of the pitfalls to the complexity brake argument oered by Allen.

We Are Our Connectome

Connectomics is an emerging eld in computational biology that seeks to do for the brain what the eld of genomics has done for the genome. A connectome is specically an annotated list of all synaptic connections within a brain (or brain region). To date, the only connectome that has been fully computed is for the worm C. elegans (Fig. 1) and consists of about 300 neurons with 7000 connections between them. Sebastian Seung famously conjectured [11] that we are our connectome, meaning that the properties of identity and consciousness that we prescribe to the human experience can eventually be accounted for by examining the neuronal connections within the brain. Acquiring such detailed images and performing the subsequent processing to yield reconstructed neural connections has proved challenging [9]. Serial sectional slices of brain are imaged with Serial Electron Microscopy (SEM) and segmentation algorithms are used to locate neurons from one serial slice to the next. If the tissue is sliced into thin layers manually, the limits of precision are about 50 nm. Since axons can be less than 100 nm thick, the task of tracing the interconnections in manually sliced data is extremely laborious and cannot be scaled to large projects. Automated slicing techniques have been devised (mostly in the last decade) for this purpose. The tissue material is rst prepared coarsely onto slides using a medium to hold it securely (such as a plastic embedding). In Serial Block Face SEM (SBF-SEM) the cutting device (an ultramicrotome) is actually placed inside of the electron microscopes vacuum chamber. The top tissue layer is imaged using the backscattered electrons and then a thin slice (usually smaller than 30 nm) is cut from the top with the diamond-tipped ultramicrotome and discarded so that the freshly revealed tissue can be imaged by the microscope. 2

(a) C. elegans slice

(b) Recovered connectome

Figure 1: Examples of worm slice and connectome graph for C. elegans. (a) Worm slice recorded in [14] (b) Graph of connections between neurons for C. elegans from [15]. This graph is produced via image segmentation and reconstruction algorithms applied to slices like (a) which ll an entire volume of tissue.

While [9] merely mentions in passing that scaling up the computer-assisted approach to larger connectomes would require armies of human operators, this really lies at the heart of the issue regarding Parberrys claims. Before we can address whether simulating all of these connections would be feasible, we have to get down to specics regarding just how dicult it would be to actually measure all of them in the rst place. Fortunately, [7] provides some extremely detailed and promising analysis that covers exactly this topic. Notably, a new (2008) procedure known as Focused Ion Beam SEM (FIB-SEM) has been developed to completely get rid of the ultramicrotome all together. In this procedure, a focused beam of gallium ions is used to ablate away layers of tissue from a block of tissue being imaged inside of the electron microscope. Ion beams can be tightly focused, which has allowed this technique to achieve an eective tissue voxel resolution of 5 by 5 by 10 nm3 . There is strong evidence surveyed in [7] that direct structural changes (size and shape) correlate with a synapses strength. Since these quantities are easily measurable in a 5 by 5 by 10 nm voxel of tissue, one could literally read out all of the connections and their strengths from the high-resolution images that result from the FIB-SEM process. The trade-o for FIB-SEM comes from the fact that the precise focus of the ion beam can only be maintained laterally over distances of about 20 microns. This means that before tissue can be automatically processed by a FIB-SEM device, it must be prepared into roughly 20 by 20 micron columns, and then rapid, high-resolution serial imaging can proceed down the column of tissue. [7] mentions a series of experiments performed to test an automated process for converting a large block of tissue into a partition of lossless 20 micron sections (note that this is a preprocessing step outside of any FIB-SEM device). The results were good, but when scaled up to the human brain, we start to see signs of trouble. Putting aside data storage, retrieval costs and the setup and preprocessing time to prepare the tissue into small enough columns, a single 20 by 20 by 20 micron tissue cube can be imaged into 5 by 5 by 10 nm voxels in roughly 2 hours, assuming a 10 MHz optimized FIB-SEM device (which is a reasonable extrapolation from current technology). It would take 30 years for one such FIB-SEM device to image a single cubic millimeter volume of tissue. The human brain is roughly 106 mm3 in volume, and so even if we parallelized a set of 100 such FIB-SEM devices and set them running continuously for 10 years, we would only have imaged 0.003% of the human brain! In summary, the current state of the art in connectomics allows us to image tissue cubes at a resolution level of 5 by 5 by 10 nm. This resolution is more than adequate to determine all interneuron connections and properties of the connection strength, but the ablating and imaging process must occur on an extremely ne scale. Preparing and processing tissue at this scale is achievable; the engineering technology already exists. Its just painfully slow. [7] makes an apt analogy with the declaration in 1961 by President Kennedy that the U.S. space program would push for getting a man on the moon by the end of the decade: connectomics seeks to land a woman in cyberspace and return her safely to consciousness. However, we are certainly more than a decade away from even being able to attempt such a thing via brain simulation. The fact that all of the engineering ingredients exist already should be solid grounds for optimism that it wont require centuries before this is achievable, but it will likely take many decades.

Are We Our Connectome?

In [7], Hayworth also discusses cognitive science models in support of the idea that we actually are our connectome as Seung suggests. While in traditional A.I. endeavors, much success has been achieved by describing how a physical device can produce goal-oriented behavior by manipulating symbols, this is perhaps not a good t for describing the way the brain operates. Feldman and Ballard [12] proposed the 100-step constraint where a computational task that cannot be performed in fewer than 100 sequential computations must not correspond to the lower-level algorithms implemented in the brain (since the brain has a ring rate of once every 10 milliseconds). It is a very exhausting thought experiment for anyone who has developed modern machine learning algorithms to imagine doing object recognition, speech processing, or other A.I. tasks in fewer than 100 sequential steps. To reconcile this apparent dierence between symbol manipulators and neural computations, the cognitive scientist Alan Newell proposed the production system framework (PSF) in 1973. Such a system consists of a block of working memory in which all of the information about the current state of a system is stored. The parts of the system that exist around the working memory all contain if-then rules, or productions. The if portion of the rule periodically checks for a syntactically matching subset of the working memory and if it ever sees such a memory match, it triggers the then clause. The key advantage of a model like this is its unordered nature. A large list of if-then triggers is capable of producing complicated, self-modifying behavior without needing to encode things in a serial, step-by-step fashion (This also appears, at least at surface level, to be more amenable to an evolutionary process. A system that relies on sequential commands, when modied, may not even compile, so to speak, let alone produce results that dont kill the organism. A PSF with a large number of rules, however, will be robust to small changes). The main result of Newells work was to produce the ACT-R model, which is considered the most comprehensive theory for human brain function [7]. Analytic results show how traditional A.I. algorithms can be implemented in this model. There is much work available on the specics of the ACT-R model and how the brain might represent the required declarative memory as well as pattern-matching circuitry. What concerns us here, however, is that the PSF idea, particularly the ACT-R model, gives us falsiable and directly testable hypotheses about how cognition might work. With access to the high resolution tissue voxels mentioned in the previous section, one could directly test for certain neural structures that are predicted by the ACT-R model. In a sense, this is the sort of abstraction that Allen calls for when saying that mind emulations require fundamentally new science. This particular model makes specic predictions that human brain software should consist of a discrete set of production rules, declarative memory, and perceptual and motor memories. These are exactly the kinds of things we can directly look for in connectomics data. This does not answer the question of whether we can directly simulate human consciousness for us, but it does do an awful lot for us. For one, it presents a road map for leveraging existing engineering technology to test certain abstractions about the nature of human consciousness. In fact, [7] addresses the economic incentive for the neuroscience community to pursue this line of research by positing a connectome observatory where stored slices of brain tissue can be called up on-demand in a random access manner, perhaps by a massively parallel array of super-FIB-SEM machines. No single neuroscientist could aord to develop this technology, but much like the way that the astronomy community pooled together resources to build large land-based and space-based telescopes, the neuroscience community could potentially do the same for a connectomics device. Through a bidding procedure, dierent brain mapping projects would compete for time on the FIB-SEM array, and the results would be highly-resolved partial connectomes of brain regions relevant to various medical and neuroscience projects. This would surely be enough to begin testing things like the ACT-R model and would presumably fuel accelerated eort into engineering the ability to resolve and store entire brains faster than 0.003% per decade. If this technology were to boom in this manner, it is not at all unreasonable to forecast rapid connectome ability within the next 50 years. We have gone on a signicant journey into the details of connectomics and looked at some speculative theories of consciousness that might be relevant. The main purpose for this analysis is to suggest that even though current connectomics imaging is too slow to give us very near-term maps of the human brain, all of the engineering ingredients exist, including testable abstractions. No part of the connectomics puzzle seems intractably dicult given enough time. In the most extreme case, if no technology improved whatsoever, the current connectomics capabilities would nish mapping the human connectome in roughly 15,000 years (assuming a larger parallel array than was considered above). But this is an exceedingly unlikely scenario. Given past examples of rapid technological development, connectomics seems to suggest

there is nothing fundamentally dierent or more dicult about brain emulation.

The success of a simulation depends on resolution and abstractions

Parberrys main premise regarding the inability to simulate brain functions is that there would be a roughly quadratic overhead. But what exactly does this mean? In formal complexity, it is not hard to understand the overhead of one Turing machines operation as a function of some resource that a dierent, simulated Turing machine requires. But when we simulate the computational model that is all of physics is this really the kind of overhead we encounter? Such an assertion that we lack computational resources to simulate a brain based on connectomics data rests on shaky semantic ground. Consider a mass-spring system. At the nest (known) level, one would need a quantum description of each of the particles involved in making up both the mass and the spring. One would need to know the curvature of space to take into account relativistic eects, however minute. One could attempt to build a simulation of a mass-spring system in this manner. When described this way, it may well be completely intractable to use a classical computer to iteratively update the state of every part of the mass-spring system. But then again, this is hardly what one generally means when one asks to simulate a physical system. Simulations are frequently bound to observational expectations. In the case of a mass-spring system, one might call the simulation faithful if it produces the correct resonant frequency or the right pattern of oscillation. These semantic attributes are not direct properties of iterative updating of each state, but rather they are a useful abstraction of subsets of the systems total state. In other words, the things we care about in simulations are generally robust to errors in potentially large numbers of specic component states. In fact, in many cases the computational overhead of simulating a physical system is dened by the complexity of the fastest classical algorithm that eectively answers the puzzling questions that the simulator has about the system. Theres no reason to avoid taking the same view to consciousness. A connectomicsbased solution that helps us cure Alzheimers will eectively explain the computational overhead for computing certain things regarding memory preservation. This same overhead may not correspond to the simulation required to simulate a better Jeopardy player than Watson, but then again, that might not be the goal of such a simulation. The main expectations of simulating a human brain are those of conscious behavior, and so the accuracy with which the entire connectome needs to be simulated will be dictated by how well the simulation performs in desired tasks. Of course, this will require some form of successful abstraction to be applied to the brains network state. But we have already seen the ACT-R model which appears to be well suited for modeling cognitive tasks borne out of the connectome. It remains to be seen whether or not we need an entire imaged connectome before direct simulation can begin to explain the mysteries of consciousness. It could just as easily be the case that small partial connectomes, when used to test the ACT-R predictions, will aord us better abstractions and insights that let us solve consciousness without needing more detailed connectome models. This would still count as a successful simulation-based approach to consciousness, though it clearly would not require the tremendous overhead that Parberry claims. Without actually imaging the brain tissue and testing the models, we wont know, and in this sense it seems that Parberrys pessimism is unproductive. While Parberrys analysis could prove to be correct in the worst case scenario that a literal simulation of each brain component is required, this hardly seems like a forgone conclusion worthy of Parberrys belief that it is the real reason why strong AI is not possible. [5] As we have seen, the engineering ability to actually image these components of the brain essentially exists today and will likely become acceptably cheap and fast within the next 100 years. With such an amazing set of imaging tools and perhaps even the connectome observatory, cognitive scientists will be able to more rapidly test abstractions like ACT-R. It seems very plausible, even likely, that models based on ACT-R (when given access to whole portions of the imaged connectome) will be able to replicate aspects of consciousness in simulation. This may prove to be untrue, but the mere fact that the brain has a large number of interconnected neurons does not seem to provide any corroborative evidence to this pessimistic view.

Belief in a complexity brake is like belief in a singularity

In addition to Parberrys computational overhead argument, we have also encountered Paul Allens complexity brake argument. Have we seen anything that conrms this complexity brake to us? When probing down to the deepest levels (the 5 by 5 by 10 nm voxel resolution) neuroanatomists suggest that the brains structure can be imaged nearly awlessly. Of course we will have to run the experiments, build the devices, incentivize their speedup, etc. But just because the work remains to be done doesnt mean it is too hard to do. In (perhaps rightful) criticism of Ray Kurzweil, Allen rejects the idea that accelerating growth is an inherent property of complex networks of agents. He treats this as an almost metaphysical mistake on Kurzweils part; the invocation of magic growth laws to achieve an optimistic outcome. But is the complexity brake idea really all that dierent? One appears to be an unjustied prior belief that growth rates will continue unabated; the other appears to be an unjustied prior belief that dicult systems have a mystical and limitless supply of complexity that we cant penetrate. It is comparable to the famous anecdote about Einsteins circle. Einstein is said to have begun a class by drawing a circle on the board and explaining that the interior of the circle represents knowledge and that as our circle of knowledge expands, so does the circumference of darkness surrounding it. But why are we restricted to drawing such a circle in a plane? If instead we are drawing it on the surface of a sphere, then there is some point in time after which the circumference of ignorance will diminish and we will be aware that we are homing in on a very successful description of some knowledge domain. Both Parberry and Allen presuppose that we are not at this stage with cognitive simulations. The connectomics evidence, however, suggests otherwise. Just as President Kennedy saw the circumference of ignorance shrinking with respect to our ability to land on the moon, largely because all of the engineering details were there and just needed to be scaled up, so too are the technical hurdles for simulating a mind all related to scaling it up. Weve stopped seeing an increase in the complexity of this problem and its now a matter of scaling up our ability to rapidly image brain tissue.

Conclusion

We do not know if simulating a brain is as straight-forward as imaging the entire connectome. There is evidence, however, that we can extract elements of consciousness by simulating parts of a connectome, and in particular by appealing to the ACT-R model of cognition. At worst, the question is merely one of whether or not humanity can provide enough ops to simulate a whole connectome. In the best case, a connectome observatory will help reveal which abstractions are useful and which are not and can be used to reduce the computational burden of consciousness signicantly. If the various estimates provided in the research literature are even remotely plausible, then we are probably looking at something like 1 to 10 decades before we have digital consciousness, rather than, say, 1 to 10 centuries. Given the impacts that this might have across many research domains it seems that, contrary to Parberrys view, A.I. through brain emulation might not be that far o and a radical reordering of research priorities across several disciplines might be appropriate. In summary we rst considered Q1: Is it feasible in an engineering sense to actually measure the characteristics of neurons and connections to such a resolution that non-trivial mental simulation could arise from them? The evidence from connectomics suggests this is absolutely feasible. In the short term we may only be able to measure parts of the connectome, but scaling computers up to the task of larger and larger connectomes will just be a matter of time. With Q2 we considered: If we can describe such a complicated brain-network state, will we have enough computational power to simulate it and observe conscious characteristics? No one can answer this question with certainty, but there is every reason to believe that if we can image large portions of the connectome, we wont need to literally simulate the entire connectome. The ACT-R model provides an example production system framework within which we can make testable claims about consciousness and perhaps discover better cognitive abstractions that will reduce the total computational burden of brain emulation. Even without successful abstraction, there is evidence [3] that raw computational power will be up to the task of simulating such a large network regardless. In [3] such successful weak-scaling is achieved with the Blue Gene super computer that the authors declare, with further progress in supercomputing, real time human-scale simulations are not only within reach, but indeed appear inevitable. In Q3 we asked: Even if we can measure the necessary characteristics of a brain network state and we have the necessary 6

computational resources, will additional abstractions be required before we can possibly make sense of (or eectively simulate) cognition from the measured brain network state? Once again, the ACT-R model and the ability to measure synapse connection strength in the connectomics data suggests that no radically new abstractions will be required. And nally, in Q4 we address Paul Allens complexity brake argument: Is there evidence of a real complexity brake preventing mind emulation? That is, does the current neuroscience of brain structure tell us that not only is a neuron-byneuron simulation insucient to recover brain function, but also that we dont yet have any idea what would be sucient? The connectomics evidence is clear: weve rounded the corner on this problem. Are there still mysteries regarding consciousness? Of course. But should we believe that we need paradigm-changing discoveries before we even have the tools to understand consciousness? Connectomics suggests not.

References
[1] N. Bostrom and A. Sandberg. Whole Brain Emulations: A Technical Roadmap. Technical Report 2008-3, The Future of Humanity Institute, Oxford University, 2008. (pdf) [2] R. Hanson. If uploads come rst: The crack of a future dawn. Extropy, 6, 1994. [3] R. Ananthanarayanan, S. Esser, and D. Modha. The cat is out of the bag: cortical simulations with 109 neurons and 1013 synapses. Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis. 2009. (pdf) [4] J. Fildes. BBC News, Articial http://news.bbc.co.uk/2/hi/8164060.stm brain 10 years away, Jul. 22, 2009. Retrived from:

[5] I. Parberry. Knowledge, Understanding, and Computational Complexity, Optimality in Biological and Articial Networks?, Chapter 8 (D.S. Levine, W.R. Elsberry, Eds.), Lawrence Erlbaum Associates, 1997. (pdf). [6] I. Parberry and G. Schnitger. Parallel computation with threshold functions. Journal of Computer and System Sciences, 36(3):278302, 1988. [7] K. Hayworth. Electron Imaging Technology for Whole Brain Neural Circuit Mapping, to appear in: International Journal of Machine Consciousness, Vol. 4, June 2012. [8] P.Allen and M. Greaves. Paul Allen: The Singularity Isnt Near, Technology Review, guest blog: Oct. 12, 2011. Retrieved from http://www.technologyreview.com/blog/guest/27206/. [9] H. S. Seung. Reading the Book of Memory: Sparse Sampling versus Dense Mapping of Connectomes. Neuron, 62, 17-29 (2009). (pdf). [10] H. S. Seung. Towards functional connectomics. Nature, Vol. 471. pp. 170-171, Mar. 10, 2011. (pdf). [11] H. S. Seung. I am my connectome, Video le retrieved from: http://www.ted.com/talks/sebastian seung.html. [12] J. A. Feldman and D. H. Ballard. Connectionist models and their properties. Cognitive Science, 6, pp. 205-254. 1982. [13] B. L. Chen, D.H. Hall, and D.B. Chklovskii. Wiring optimization can relate neuronal structure and function, PNAS, 103:12 p4723-28. 2006. [14] This image was taken from The WormImage Database, http://www.wormimage.org/index.php. This specic image can be found here: http://www.wormimage.org/image.php?id=16019&page=1. [15] This image was taken from BrainMaps.org,http://brainmaps.org/index.php?p=brain-connectivity-maps, and reused in this paper under the Creative Commons Attribution 3.0 License, http://brainmaps.org/index.php?p=termsofuse. The image was originally produced via the data found in [13] and the C. elegans connectome data is now publicly available from the Open Connectome Project.