Metaconsciousness: Mythology for a Post-Civilized World
Chapter 6 | Contents | Chapter 8
As mentioned in the Prologue, Daniel Quinn cites the circus as an example of a contemporary tribe that functions effectively within the environment created by civilization.1 I would like to suggest another, more spectacular and globe-girdling example: the free software / open-source community, or the tribe of hackers.2
Hackers, by the way, are justifiably irritated by the pejorative and erroneous spin the "meanstream press" have attached to their chosen sobriquet. Just to set the record straight, a hacker is authoritatively defined as
1. A person who enjoys exploring the details of programmable systems and how to stretch their capabilities, as opposed to most users, who prefer to learn only the minimum necessary. 2. One who programs enthusiastically (even obsessively) or who enjoys programming rather than just theorizing about programming. 3. A person capable of appreciating hack value. 4. A person who is good at programming quickly. 5. An expert at a particular program, or one who frequently does work using it or on it; as in "a Unix hacker". (Definitions 1 through 5 are correlated, and people who fit them congregate.) 6. An expert or enthusiast of any kind. One might be an astronomy hacker, for example. 7. One who enjoys the intellectual challenge of creatively overcoming or circumventing limitations. 8. [depreciated] A malicious meddler who tries to discover sensitive information by poking around. Hence password hacker, network hacker. The correct term for this sense is cracker.
The term "hacker" also tends to connote membership in the global community defined by the net.... It also implies that the person described is seen to subscribe to some version of the ... hacker ethic.
It is better to be described as a hacker by others than to describe oneself that way. Hackers consider themselves something of an elite (a meritocracy based on ability), though one to which new members are gladly welcome. There is thus a certain ego satisfaction to be had in identifying yourself as a hacker (but if you claim to be one and are not, you'll quickly be labeled bogus.3
Hackers invented, built, and populate the World Wide Web.4 Hackers have "reverse-engineered" the proprietary computer operating system Unix, originally developed by Bell Labs hackers Ken Thompson and Dennis Ritchie, and have given to the world the free / open-source GNU / Linux operating system which practically runs the Internet today, and is probably more powerful, functional, stable, and certainly more cost-effective, than any proprietary operating system now available for the expanding global population of personal computer users.
In The Cathedral & the Bazaar5 Eric Raymond provides considerable insight into the particulars of hacker culture, as well as a penetrating analysis of the astonishing success of the "bazaar style of software development" exemplified by the rise of Linux; in comparison to the "cathedral style" of proprietary software development. I submit that valuable lessons may be learned from a study of these disclosures, and that they are applicable to a far wider field than that of pure software development.
In particular, the global hacker tribe have demonstrated a remarkable agility in dealing with civilization, not by opposing it, but by doing, on an expanding and accelerating scale, essentially what I am advocating in this book, i.e. fomenting metaconsciousness. It is, in fact, "written into" the hacker ethic, which Raymond defines as:
1. The belief that information-sharing is a powerful positive good [emphasis added], and that it is an ethical duty of hackers to share their expertise by writing free software and facilitating access to information and to computing resources wherever possible. 2. The belief that system-cracking for fun and exploration is ethically OK as long as the cracker commits no theft, vandalism, or breach of confidentiality.
Both of these normative ethical principles are widely, but by no means universally, accepted among hackers. Most hackers subscribe to the hacker ethic in sense 1, and many act on it by writing and giving away free software. A few go further and assert that all information should be free and any proprietary control of it is bad; this is the philosophy behind the GNU project.6
Sense 2 is more controversial: some people consider the act of cracking itself to be unethical, like breaking and entering. But the belief that 'ethical' cracking excludes destruction at least moderates the behavior of people who see themselves as 'benign' crackers.... On this view, it may be one of the highest forms of hackerly courtesy to (a) break into a system, and then (b) explain to the sysop, preferably by email from a superuser account, exactly how it was done and how the hole can be plugged – acting as an unpaid (and unsolicited) tiger team.
The most reliable manifestation of either version of the hacker ethic is that almost all hackers are actively willing to share technical tricks, software, and (where possible) computing resources with other hackers. Huge cooperative networks such as Usenet, FidoNet and Internet ... can function without central control because of this trait; they both rely on and reinforce a sense of community that may be hackerdom's most valuable intangible asset.7
Raymond's analysis of the utility of open-source8 software is careful to step around the "ideological" proposition that "information should be non-proprietary." Whether it "should" or "shouldn't," there are qualitative pros and cons that have "real-world" impact upon a product's utility, and these have been widely misunderstood.
For example, the idea behind making a creative work proprietary is ostensibly to deny one's competitors the ability to duplicate and profit from the work at its author's expense. One can see the thing from the author's point of view. A manufacturer, for instance, usually does not welcome a competitor selling the same product under a different brand name, especially when he, the original manufacturer, made the investment necessary to develop the product and bring it to market. One can appreciate how such a person might reasonably take measures to keep his product proprietary and exclusive to himself and his designated (paying) licensees.
There are important differences, however, between software, and possibly of other kinds of information, and "manufactured goods." Contrary to widespread belief, software developers do not primarily manufacture software: they provide a service. Their service only begins with the acquisition of a software product by a client. In order to keep the client happy, the developer must maintain the product, make it as adaptable as possible to the client's unique needs, repair its flaws (debug the program), and provide enhancements and improvements throughout the useful lifetime of the product. A developer who conscientiously and reliably provides these services will out-perform a developer who does not. Such services can be time-consuming and costly, however, and if the developer treats his product as a manufactured good seeking a one-time sale, he is going to loose in the long run to his competitor who operates on the basis of a different model, other factors being equal.
The "bazaar style of software development" has turned out to have been such a "different model;" as opposed to the "cathedral style," which treats software, in part, as a manufactured good for sale. In 1991 a Finnish hacker named Linus Torvalds began work on a non-proprietary clone of the Unix kernel for Intel 386 processors. At the time, nobody had dreamed that a lone hacker, or even a team of hackers, could produce anything as complex as the kernel – the heart, the very core – of a functional operating system. Such "high end" projects, it was universally believed, could only be successfully developed by well-organized teams of highly trained professionals – which of course could only be assembled in the milieu of hierarchically structured corporate entities. Linus's project, however, attracted the voluntary participation of large numbers of other hackers, just because it was so cool, and by the end of 1993 Linux was competitive in reliability and stability with many proprietary Unix flavors, and supported an enormously larger software base – including even some commercial applications.
What was happening, and has continued to happen, was the synergistic convergence of a highly appealing project among hackers, with the sudden emergence of the Internet into the "mainstream" via the World Wide Web. This brought to bear the creative ingenuity of thousands of relatively isolated hackers from around the world, and the "impossible" emerged as the increasingly functional and robust "free software" product, GNU / Linux.9
From nearly the beginning [Raymond writes], [Linux] was rather casually hacked on by huge numbers of volunteers coordinating only through the Internet. Quality was maintained not by rigid standards or autocracy but by the naively simple strategy of releasing every week and getting feedback from hundreds of users within days, creating a sort of rapid Darwinian selection on the mutations introduced by developers. To the amazement of almost everyone, this worked quite well.10
No one was more thoroughly flabbergasted by the performance of the GNU / Linux operating system than longstanding GNU hacker Eric Raymond.
Linux overturned much of what I thought I knew [he writes]. I had been preaching the Unix gospel of small tools, rapid prototyping and evolutionary programming for years. But I also believed there was a certain critical complexity above which a more centralized, a priori approach was required. I believed that the most important software (operating systems and really large tools like the Emacs programming editor) needed to be built like cathedrals, carefully crafted by individual wizards or small bands of mages working in splendid isolation, with no beta to be released before its time.
Linus Torvalds's style of development – release early and often, delegate everything you can, be open to the point of promiscuity – came as a surprise. No quiet, reverent cathedral-building here – rather, the Linux community seemed to resemble a great babbling bazaar of differing agendas and approaches (aptly symbolized by the Linux archive sites, which would take submissions from anyone) out of which a coherent and stable system could seemingly emerge only by a succession of miracles.
The fact that this bazaar style seemed to work, and work well, came as a distinct shock. As I learned my way around, I worked hard not just at individual projects, but also at trying to understand why the Linux world not only didn't fly apart in confusion, but seemed to go from strength to strength at a speed barely imaginable to cathedral-builders.11
Accordingly, Raymond undertook the ethnological study of how the hacker tribe that formed around Linux actually worked. Raymond wrote up his four-year analysis in a paper titled "The Cathedral and the Bazaar," first delivered publicly in May, 1997, at the Linux Kongress in Bavaria, and reproduced in his book of the same title.12
The hacker tribe greeted Raymond's analysis with thunderous applause, for he had given them an image of themselves, and a picture of what they were doing, and its significance, of which they themselves had been virtually unaware. The real kicker came eight months later, however, when Netscape Communications, Inc. announced their decision to go open-source with their line of Netscape browsers; and their CEO Jim Barksdale was citing Raymond's paper to the media as the "fundamental inspiration" for their decision.
This was the event [Raymond writes] that commentators in the computer trade press would later call "the shot heard 'round the world" – and Barksdale had cast me as its Thomas Paine, whether I wanted the role or not. For the first time in the history of the hacker culture a Fortune 500 darling of Wall Street had bet its future on the belief that our way was right. And, more specifically, that my analysis of 'our way' was right.13
Well, good on Eric Raymond! Yet he was sobered by the implications of the fact that someone from the heart of the "mainstream" had gone out on a limb to follow the path through the jungle being hacked by the hacker tribe. Up front, it looked like a big feather in the hacker cap – but what if Netscape's gambit failed? This was a significant possibility, for the colossus and prototypical "cathedral-builder" Microsoft had marked Netscape down for its prey; and victory for Microsoft in this contest would not do the hackers one bit of good.
As we'll be seeing in greater depth in the next chapter, Microsoft's strategy has been to box their market exclusively into reliance upon Microsoft products by "embracing and extending" universal protocols in such a way as to turn them into de facto Microsoft protocols – so that only Microsoft proprietary software can use them. This is exactly counter to the original intent of Tim Berners-Lee, inventor of the HTTP and HTML protocols which support the World Wide Web;14 and it is anathema to hacker culture.
For Netscape [Raymond writes], the issue was less about browser-related income (never more than a small fraction of their revenues) than maintaining a safe space for their much more valuable server business. If Microsoft's Internet Explorer achieved market dominance, Microsoft would be able to bend the Web's protocols away from open standards and into proprietary channels that only Microsoft's servers would be able to service.15
_____________________________________
1. Quinn, 1999.
2. Eric S. Raymond, The Cathedral & the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary, O'Reilly, Beijing, Cambridge, Farnham, Köln, Paris, Sebastopol, Taipei, Tokyo, 1999.
3. The New Hacker's Dictionary, Third Edition, compiled by Eric S. Raymond, The MIT Press, Cambridge, Massachusetts, London, England, 1996, 1998, pp. 233-4. Note that boldface words in reference to The New Hacker's Dictionary refer to entries in the Dictionary (also called the yellow book), on-line at
www.catb.org/~esr/jargon/html/H/hacker.html.
4. J. Harmon Grahn, "The World Wide Web," The New Paradigm, vol. III, #4, 1/26/2000.
5. Raymond, 1999. [on-line at www.catb.org/~esr/writings/cathedral-bazaar/].
6. The GNU project. It is also in approximate harmony with the philosophy behind the Freedom Digital Library. See our Draft Vision Statement, § 4.2, for elaboration. See also Martin, 1995. I'll add here the idea that any author is entitled by free choice to make his or her work proprietary, just as anyone is entitled to have and to keep secrets; and is conversely responsible to take whatever measures he or she can to keep it so. If the work of Dr. Hawkins, described in Chapter 4, is widely applied, it may not even be possible to "keep a secret." It is not in any case properly incumbent upon any third party to keep an author's work proprietary, or to keep his secrets, absent explicit person-to-person agreement to do so – for the reason that all preemptively "legislated" obstructions to the free flow of information are by nature obstacles to the expansion of metaconsciousness, and are consequently stifling to the evolution of Life in Cosmos; and all Life is naturally entitled to defend itself from any entity or agency that threatens or stifles it.
7. Raymond, The New Hacker's Dictionary, pp.234-5.
8. See the Open Source Definition at www.opensource.org/docs/definition_plain.html.
9. These developments are related in "A Brief History of Hackerdom" Raymond, 1999, pp. 23-5. "Linux" should most properly be called "GNU / Linux," as Linus built his kernel around the GNU system under development at the Free Sofware Foundation. And, by way of a hint to those who would number themselves among the hacker cognoscente, "Linux" should most properly be pronounced "Leenuks," in consistency with the way Linus pronounces his name: "Leenus," not "L eye nus."
10. "A Brief History of Hackerdom," Raymond, 1999, p. 24.
11. "The Cathedral and the Bazaar," Raymond, 1999, pp. 29-30.
13. "The Revenge of the Hackers," Raymond, 1999, p. 203.
15. "The Revenge of the Hackers," Raymond, 1999, p. 202.
Metaconsciousness: Mythology for a Post-Civilized World
Chapter 6 | Contents | Chapter 8