Wednesday, April 16, 2014

The Delightful World of Open Source Software

The Heartbleed vulnerability was an awful bug, and caused incredible damage to the web of trust we have built around online transactions. But, there was a silver lining: people became aware of how much of our critical Internet structure relies on old, poorly-maintained code that nobody wants to support any more. That's, um, a bad thing, but now that we're aware of it people can start fixing it!


By far my favorite thing I've seen online recently is the git commit log for (a copy of) OpenSSL, the software with the Heartbleed bug. Once people started paying attention, interest in the project spiked, and they started seriously working to not just plug Heartbleed, but to fix the code in general. This led to a flurry of activity, and a fascinating kind of oral history that reminds me of someone discovering the Necronomicon: initial excitement and a sense of superiority gradually give way to a creeping suspicion that something is not right here, and ending in a howling wave of madness as everything you love in the world is destroyed. Here are some of my favorite commit messages, all from the last three days or so, presented in chronological order:

I am completely blown away that the same IETF that cannot efficiently allocate needed protocol, service numbers, or other such things when they are needed, can so quickly and easily rubber stamp the addition of a 64K Covert Channel in a critical protocol.  The organization should look at itself very carefully, find out how this this happened, and everyone who allowed this to happen on their watch should be evicted from the decision making process.  IETF, I don't trust you.
 


remove more cases of MS_STATIC, MS_CALLBACK, and MS_FAR. Did you know that MS_STATIC doesn't mean it is static?  How far can lies and half-truths be layered?  I wonder if anyone got fooled, and actually returned a pointer..

 

Remove various horrible socket syscall wrappers, especially SHUTDOWN* which did shutdown + close, all nasty and surprising.  Use the raw syscalls that everyone knows the behaviour of.

 

First pass at applying KNF to the OpenSSL code, which almost makes it readable.




Flense all use of BIO_snprintf from ssl source - use the real one instead


o_dir.c has a questionable odor


Toss a `unifdef -U OPENSSL_SYS_WINDOWS' bomb into crypto/bio.

 

No longer mention OPENSSL_EC_BIN_PT_COMP being required to allow for `compressed' EC point representation.
First, as researched by djb, quoting from http://cr.yp.to/ecdh/patents.html :
``It should, in any case, be obvious to the reader that a patent cannot
  cover compression mechanisms published seven years before the patent
  was filed.'' 

Second, that define was actually removed from the code in in OpenSSL 1.0.0.

 

remove FIPS mode support. people who require FIPS can buy something that meets their needs, but dumping it in here only penalizes the rest of us.

 

Q: How would you like your lies, sir?
A: Rare.


 

just like every web browser expands until it can read mail, every modular library expands until it has its own dlfcn wrapper, and libcrypto is no exception.
 


The NO_ASN1_OLD define was introduced in 0.9.7, 8 years ago, to allow for obsolete (and mostly internal) routines to be compiled out. We don't expect any reasonable software to stick to these interfaces, so better clean up the view and unifdef -DNO_ASN1_OLD. The astute reader will notice the existence of NO_OLD_ASN1 which serves a similar purpose, but is more entangled. Its time will come, soon.

 

imake died in a fire a long time ago

 

we don't use these files for building

 

we don't use this makefile

 

the VMS code is legion 


remove ssl2 support even more completely. in the process, always include ssl3 and tls1, we don't need config options for them. when the time comes to expire ssl3, it will be with an ax.


Remove wraparounds for operating systems which lack issetugid(). I will note that some were missing, looking at you Solaris!!!  Anyone home? Using my own copyright on the file now, since this is a rewrite of a trivial wrapper around a system call I invented.


use explicit_bzero instead of a bizarro "no compiler could ever be smart enough to optimize this" monstrosity.

 

Three wrappers in this file: OPENSSL_strncasecmp, OPENSSL_strcasecmp, and OPENSSL_memcmp. All modern systems have strncasecmp.  No need to rewrite it. Same with memcmp, call the system one!  It is more likely to be hot in the icache, and is specifically optimized for the platform.  I thought these OpenSSL people cared about performance?

 

you do not want to do the things this program does




strncpy(d, s, strlen(s)) is a special kind of stupid. even when it's right,it looks wrong. replace with auditable code and eliminate many strlen calls to improve efficiency. (wait, did somebody say FASTER?)


spray the apps directory with anti-VMS napalm. so that its lovecraftian horror is not forever lost, i reproduce below a comment from the deleted code.
        /* 2011-03-22 SMS.
         * If we have 32-bit pointers everywhere, then we're safe, and
         * we bypass this mess, as on non-VMS systems.  (See ARGV,
         * above.)
         * Problem 1: Compaq/HP C before V7.3 always used 32-bit
         * pointers for argv[].
         * Fix 1: For a 32-bit argv[], when we're using 64-bit pointers
         * everywhere else, we always allocate and use a 64-bit
         * duplicate of argv[].
         * Problem 2: Compaq/HP C V7.3 (Alpha, IA64) before ECO1 failed
         * to NULL-terminate a 64-bit argv[].  (As this was written, the
         * compiler ECO was available only on IA64.)
         * Fix 2: Unless advised not to (VMS_TRUST_ARGV), we test a
         * 64-bit argv[argc] for NULL, and, if necessary, use a
         * (properly) NULL-terminated (64-bit) duplicate of argv[].
         * The same code is used in either case to duplicate argv[].
         * Some of these decisions could be handled in preprocessing,
         * but the code tends to get even uglier, and the penalty for
         * deciding at compile- or run-time is tiny.
         */

 

Remove non-posix support. Why is OPENSSL_isservice even here? Is this a crypto library or a generic platform abstraction library? "A hack to make Visual C++ 5.0 work correctly" ... time to upgrade.
 


Your operating system memory allocation functions are your friend. If they are not please fix your operating system.

 

Make this byzantine horror a shell of it's former self by stubbing the functions. The ability to set the debug mem functions died with mem.c


Actually, now that I look at all that together at once, it's really reminding me of Johnny's journal entries from House of Leaves.

Now, for an open-source enthusiast like myself, the entire Heartbleed saga has been distressing on a psychological, even philosophical level. One of the core axioms that open-source advocates embrace is the belief that open software leads to greater security and stability in code. When you offer up your source code for the entire world, you gain thousands of eyeballs, any of who can spot bugs in the code and offer solutions. The thinking is that this is much safer than the closed-source Microsoft world, where bugs lie hidden in compiled code, and can't be discovered until a nefarious actor exploits it.

Heartbleed totally inverted that expectation: all the lame companies who used Microsoft IIS came out of the incident with flying colors, while all of the cool companies running LAMP-style stacks seemed like dupes. The truly embarrassing thing is that the bug was in OpenSSL for over two years before it was finally noticed and fixed. That seems to disprove Eric Raymond's Bazaar argument that "given enough eyeballs, all bugs are shallow."

Now that the dust has settled somewhat, a more nuanced view seems to be emerging. The reality is that there were almost no actual eyeballs on OpenSSL; even though six billion people could have looked at it, only three people were spending a couple of hours a month. Why? Well, because:
  1. It's ancient software. Programmers are always excited by newer and better things; who wants to waste time trawling through outdated code?
  2. It's a nightmare to read. As Neal Stephenson noted in his chapter on Linux from In the Beginning..., most low-level open-source software is written in C, and contains such a staggering amount of boilerplate precompiler definitions that it's a nightmare just to find where the actual code in the project lives. Many or most of the above-quoted commits are related to this complaint.
  3. Tied to the first two points, virtually all open source software is maintained on a purely volunteer basis. These are almost always programmers with full-time day jobs, so they'll spend their limited free-time programming hours on software that excites them and/or that has potential for future career advancement; which, respectively, means well-written projects and/or projects using cutting-edge technology. Exciting new projects with clean code and active communities like Django get a lot of volunteers and can advance very quickly; fugly old legacy projects like OpenSSL don't get volunteers.
  4. So, because nobody wants to work on these projects that are critically important but mind-numbingly dull, it's up to "the community" to fund continued support. Last year, the project received $2000 in donations, which isn't much at all, and most of which came from a couple of Internet companies. That's wildly out of sync with how important the software is, and a staggeringly small amount of resources from the Internet companies (Google, Amazon, Facebook, etc.) that rely on using the software. (By my calculations, the total budget last year for OpenSSL is equivalent to roughly 1.5% of a single Google engineer's salary.)
Fundamentally, Raymond isn't wrong, but we were wrong to assume that just because a limitless number of people could review software, enough people were reviewing it. 

It will be interesting to see how the industry as a whole responds to this incident. In the short term, I'm concerned that we might see a surge of similar exploits: now that it's widely known that vulnerabilities can languish for years in these open-source projects, criminal hackers are probably poring through CVS repositories looking for the next unpatched buffer overflow exploit. Any such additional vulnerabilities would intensify a call to action, but one is probably inevitably coming anyways. What needs to happen? In my opinion, those who have profited most from the labor of the community should contribute more to its survival, either fiscally (via donations of money) or in kind (by assigning employees to their maintenance, as Google currently does). Hopefully self-interest will motivate the big companies to do so; if not, it may be helpful to "name and shame" any freeloaders. (And, really, we're not talking about huge sums of money here, certainly far less than paying a license for commercial software would cost.)

The other big thing that needs to happen is what those wonderful, doomed folks at OpenSSL are doing now: throwing back the curtain around this old software, gaping in unfeigned shock at how awful it is, and taking a chainsaw to the worst bits of it, trying to hack it down to a state that's possible to comprehend and maintain. Any software developer will tell you that this is the worst kind of programming imaginable: fixing bad code that you didn't write in the first place. But, it's crucially important. (I'll be curious to see if this incident also adds a sense of urgency for rewriting software components in more modern languages that don't even allow frequently-exploited "features" like buffer overflows.)

With that in mind, I'm even happier to read the words in this commit log. Not just because they're funny, not just because they're enlightening, but because they're a part of the long oral tradition that is open source software. More so than any other area of software development, open source relies on programmers being brutally candid about what's going on: if someone writes a hack, you can be sure that they'll leave a comment pointing out that it is a hack, explaining why they had to do it, and what the implications of that hack are. Without any corporate PR arm around to vet them, coders can be perfectly frank about their opinion of any software that they're writing or reading. Back when I was in college and first getting into Linux, whenever I was bored (and didn't feel like piping random text files to /dev/dsp) I would open up a console prompt and type grep -R <my favorite curse-word> /usr/src/linux. This would inevitably bring me to the most interesting parts of the Linux codebase: not the reams of #ifdef macro commands, not the fiddly make settings for obscure hardware, but the places where some philosophical debate was occurring between different generations of Linux developers. Heartbleed is almost certainly the biggest threat that open source software has faced in the past twenty years, but if it can quickly respond with transparency, candor, and action, it will emerge stronger than ever before.

Saturday, April 12, 2014

Ashes, Ashes, We All Fall Down!

Without really planning it, I've ended up going on a bit of an Obsidian kick. After wrapping up Mask of the Betrayer, I finally started playing Fallout New Vegas, which has been sitting in my Steam library unplayed for a distressingly long time. (As always, mega thanks to my youngest brother for a seemingly endless bounty of amazing games that I have yet to play.) As with most games (and books for that matter), there's no good reason why it took me this long to get started, other than the sheer quantity of other things that I wanted to play.

It's pretty well established by now that Obsidian occupies a very interesting niche as a developer. They've become most famous for continuing franchises that were started by other developers: they made the sequels to Knights of the Old Republic, Neverwinter Nights, and later installments of Fallout and Dungeon Siege. In most cases, the mechanics of the games are very similar to their predecessors; however, there is still an identifiable style that unites their games, with an emphasis on surprising stories, memorably flawed characters, and varied game endings.

Fallout is a bit of an interesting example: it immediately appears to be another sequel job, but in fact, it's arguably more a case of the game returning to its roots. The original Fallout was developed by Black Isle/Interplay back in the 90s, and when Bethesda bought the rights to the series after Interplay folded, many gamers were outraged at the thought that their beloved turn-based isometric game was being rebooted as a real-time first-person shooter. However, many of the old Black Isle/Interplay developers ended up at Obsidian (the remainder largely settling at inXile), so New Vegas was actually a chance for many of the original Fallout creators to return to their universe.

And, I have to say, it's a pretty triumphant return. Yes, graphically, it at first glance appears identical to Fallout 3, with the game generally first-person camera, Pip-Boy menu system, radio stations, etc. It also keeps the best aspects that Bethesda added to the franchise, with its insanely detailed and vast open world, with total freedom to go anywhere at any point in the game. As you dive deeper into it, though, you begin to see the Obsidiany awesomeness hidden within. The single best improvement is probably a complete overhaul of the "morality" system, which was probably my least-favorite aspect of Fallout 3. Like the BioShock games, this used to just be based on a straight "Karma" tracker that went up when you did good things and down when you did bad; also like BioShock, there wasn't any reason to ever think about a specific choice, and the game mechanically encouraged you to just keep doubling down on rescuing kittens or punching babies as your morality dictated. New Vegas keeps the idea of Karma around, but the moral landscape is much more varied and interesting. There aren't just "the good guys" and "the bad guys," but a variety of factions and philosophies vying for superiority in the wasteland, and it's often quite challenging to decide what to do. Will you subvert the course of justice in order to allow a greater good to take place? Will you lose Karma by breaking into someone's home, knowing that the evidence inside will exonerate an innocent man? Will you kill prisoners of war in order to deny their captors the psychological leverage they crave?


One of the coolest things, which didn't become clear to me for a while, was that this is also the rare game that actually depicts that varied individuals that make up institutions. It's almost automatically assumed now that every member of a video game faction will share all of that faction's characteristics: a paladin in the Order of the Radiant Heart is always generous, kind, and law-abiding; a member of House Hlaalu is always crafty; businessmen in Rapture are always amoral and greedy. New Vegas, while still presenting some solid generalizations about the various factions, also presents outliers within those factions: and not necessarily traitors or reformers, either, just people who, due to their life history or objectives, have decided to join an institution, but have their own ideas about the actions they should take within it.

It's getting hard to keep writing about this without getting into spoilers, so let me take a step back first and cover some purely technical aspects of the game. First, the good:
  • I like the way the reputation system is handled and tuned. Like the earlier Fallout games, you have a different reputation among all the various factions and cities, so you might be Idolized on the Strip, Accepted by Novac, Shunned by the Brotherhood, and Hated by the NCR. This affects the ambient dialogue you hear, and also affects the prices you receive from merchants. Reputation is scaled quite nicely based on the size and importance of each location. You don't need to do very much to impress the small community of Novac, but it will take a very long time and a lot of effort to become famous within the NCR.
  • The economy is great! It's so refreshing to see a game built on the Elder Scrolls platform that makes good use of money. Early in the game, ammo is scarce, and it's worthwhile to scavenge pieces of junk that you can sell or recycle for bullets. As you grow richer, it stops being worth your time to poke through trash, and you'll only focus on more valuable items. You can find plenty of good equipment by exploring, but there are very valuable Implants you can buy that are pricey and worth saving up for, ranging from 4,000 caps all the way up to 12,000 caps. Throughout almost the whole game, earning money is exciting, because you're getting closer to another upgrade. And, once you've bought everything, you can still spend caps to Repair unique weapons and armor, or to buy vanity upgrades for your housing.
  • Along the same lines, I'm really happy with how the Fallout games in general approach looting. Over the years, my tolerance for loot systems in RPGs has drastically declined: I used to think nothing of sifting through my backpack for ten minutes, trying to pick the perfect combination of weight and bulk that I could carry back to town to optimize my payout. These days, I practically weep for joy at inventory systems like that in Dragon Age 2, which drastically streamlines inventory management and looting. Fallout is, in every way, a callback to old-school looting, with vast quantities of odds and ends that serve no purpose but to be sold for a few coins. And yet, I love it, because it's so perfectly attuned to the setting and feel of the game. This is, after all, a post-apocalyptic game, where a fraught present is lived amidst the detritus of a failed past. This is an entire world that's basically lost the capability to build new things, and everyone who has managed to survive, has done so by picking over the remnants of long-gone civilization. When you dig through the charred remnants of a library, you're not just looking for money: you're participating in the life of a "prospector", feeling and acting out the activities of everyone else on this planet. It's another example of gameplay, environment, and story aligning in a nicely compelling way (though more subtle than something like, say, the Spirit Eater). 
  • The ambient music for the game is really good. It's atmospheric and subtle, and does a great job at establishing a sense of place and an appropriate level of tension. (Nested criticism: while I like the idea of the old-timey radio stations, I could never stand to listen to any of them for longer than a song or two. I love hearing the news reports, but always found the ambient music much better than the retro radio stations.) My favorite was probably the music for Hoover Dam, which is really exhilarating and gives a great sense of urgency to the game.

Then, the bad:
  • VATS seems less useful than in Fallout 3. It's admittedly been  many years since I played that, but I'm pretty sure that your actions whilst in VATS were pretty much instantaneous, so you could line up multiple shots in succession, and also not need to worry much about getting hit. In NV, time runs in slow-motion, but you're in slow-motion too, so you don't really gain much. And, since other people are moving, your target might move behind cover, or an ally might run into your line of fire, causing you to waste AP. That's all annoying, but I felt especially bummed to have "wasted" several valuable perks specifically on VATS-related features, which ended up being much less useful than I had hoped.
  • Dialogue is generally much better than that in Bethesda's other titles, and there isn't anything as mind-shatteringly annoying as the "Arrow in the knee" comment in Skyrim. But, there's still the fundamental problem of far more characters present in the world than recorded dialogue, which leads to recycling. It was particularly perplexing that they decided to record the exact same lines for so many voices, though... for example, walking through a particular stronghold, you might hear a generic white woman say "I hear that Mr. House runs the strip". Then, ten seconds later, you'll pass a generic white man say "I hear that Mr. House runs the strip". Then shortly after that you'll pass a generic black man who says "I hear that Mr. House runs the strip". As long as they were going to the bother of recording three different actors providing ambient dialogue, why not at least give them different dialogue to say?
  •  While the non-ambient dialogue is generally well-written, this game is lightyears behind BioWare's engine in depicting conversations. Other characters in the frame will freeze awkwardly in whatever pose they happened to be in, and the speaker's body stands stock-still while their eyes and mouths move around. It's not quite as uncanny-valley as some other games out there, but still seems pretty dated.
  • Voice acting is generally better than most Bethesda games, but still has the problems with consistency in tone and volume that I've come to expect. In one particularly bad case, it seems like there was no agreement in the recording studio on how to pronounce the name of one major character, "Caesar". Many major characters pronounce it in the Latin style, as "Kai-sar", while most minor characters pronounce it as we do today, "See-zer". They attempted to retcon this by adding a character in the first town who says something like, "Boy, nobody knows how to say his name!" but it's still pretty funny and embarrassing.
  • This game probably has the most psychotically wide variety of keystrokes used to cancel a screen. Depending on which screen you're looking at, sometimes you need to press "X", sometimes "E", sometimes "Tab", sometimes "Esc", and sometimes need to click the right mouse button. It's especially perplexing because the interface is so obviously tuned for consoles, which don't have nearly as many buttons available as PCs, so I don't know why they decided to go so absurdly wild on mixing up the control for "I don't want to do this please."
  • And, this is a perennial complaint for any RPG with both PC and console ports, but the menus are far simpler than they need to be. In particular, trying to find stuff to sell a merchant is very time-consuming. The barter interface does give an option to only show certain categories of items (Weapons, Armor, etc.), which is helpful, but it would be extremely handy to be able to sort items by value and by value/weight ratio.
For character build: I was pretty happy with the character generator, which, in addition to the now-requisite choice between male or female, also gives you templates for several different races, including Hispanic and Asian characters. I ended up creating a female Asian named Hessie, and gave her bright red hair, vaguely thinking of going for a Run Lola Run look. I had a tough time deciding how to allocate my stats: I had played a very stealthy character in F3, which would require a lot of Agility and some Perception; I liked the idea of creating a Charismatic leader; and I really wanted to play an Intelligent character who could hack things. I eventually modified my initial character concept, from a stealthy sniper type to more of a science geek: Intelligence would help me hack terminals, a very common and useful activity, but also helps use Energy weapons, which aren't quite as useful for sniping but very powerful.


In addition to tagging skills, you can also pick some unique character options that are only available at creation. I picked up one called Four-Eyes, which lowers your Perception unless you're wearing glasses; with glasses, you gain a bonus. I had initially thought that this would be all upside: there are plenty of glasses lying around, and no penalty to wearing them. But, it turns out, the way this is implemented is that it actually permanently lowers Perception by 1 point, then gives you a +2 bonus when you have glasses on. That doesn't seem like that big of a deal, except it meant that I wasn't able to get the Perks that required a higher Perception, since Perks only pay attention to your un-modified stats. I ended up needing to "waste" a perk on the Perception upgrade in order to get some of the combat-related perks that I was planning to take. The "Small Frame" trait, though, was pretty straightforward, giving me a free Agility point at the cost of some Carry Weight. I really liked the idea of playing a small nerd with bad eyesight trying to survive in post-apocalyptic America.

Here are some general gameplay tips that may be of use:
  • As in Fallout 3, Lockpicking and Science are very important for opening locked doors and terminals, respectively. Early on you might want to focus more on Science, since in many cases a locked door can alternately be opened via a terminal. But, you'll probably want to level up both fairly early on, since the best stuff is usually kept behind locks.
  • One big thing that I'm pretty sure is new in New Vegas is skill magazines. These are single-use consumable items that give +10 to a given skill for about a minute. They can be found lying around or bought from some merchants. For things like Lockpicking and Science, it can be very handy to just bring your skills up to within 10 points of a threshold, and then use a Magazine to hit it. So, instead of targeting 25, 50, 75, and 100 points in each, try to get each up to 15, 40, 65, or 90. If you have clothing that gives bonuses, that can be taken into account as well: with a coat that gives +5 Science, you could shoot for 10, 35, 60, or 85. Anyways, the point is, this will free up valuable Skill Points to use in skills that are used more frequently (like Guns), while still being able to open the things you need.
  • Science and Lockpicking are, as noted above, generally only tested at multiples of 25, so values in between generally aren't useful. So, for example, it would be much more useful to level up as (0, 0, 0, 10, 15) or (10, 15, 0, 0, 0) than as (5, 5, 5, 5, 5). In other words, when you're ready to raise a skill, dump all you can in to get it to the next level, instead of gradually raising it.
  • In contrast, most other skills are tested at finer levels. In particular, there are a lot more conversation checks than in F3. Speech is still probably the most frequent, but tons of conversations will now have options that can be unlocked with things like Barter, Healing, Medicine, or (once again) Science. These can be tested anywhere in 5-point increments, so gradual improvements are useful. Stats (like Intelligence or Charisma) are also sometimes useful, as are particular perks.
  • While magazines and clothing are useful for raising skills for tests, note that Perk prerequisites are always determined by the base, unmodified skill level. So, if there are particular Perks you want to get, plan in advance so you can hit them by the right time. (If you want to get them immediately, that is. There's nothing wrong with waiting for a few levels later to pick something up.)
  • The Endurance stat determines how many Implants you'll be able to install. Most Implants will raise another stat (like Perception or Intelligence) by a single point. So, it can be a really good idea to get a lot of Endurance when building your character, since you'll eventually be effectively doubling each point you put into it.

MINI SPOILERS

So, here's basically how my game went down:

I left Goodsprings a little prematurely: I did most of the quests, but didn't have high enough Explosives for the last pre-requisite to prepare for the shootout with the prison gang. I'm guessing that it would have turned out OK with all the rest of the preparation I'd done, but I'm enough of a completist that I didn't want to risk it. I kept planning to come back, but somehow never found an Explosives magazine in the whole game, so as far as I know that merchant is still hiding out in the gas station.

I'd read online that it's a good idea to get to the New Vegas Medical Clinic soon, so you can get the Intelligence implant and the extra Skill Points it provides. I was warned off from the northern highway, which was overrun by Deathclaws, so I ended up swinging far to the south and then looping up through the east side. Along the way I stopped at the 188 Trading Post, where I met Veronica, chatted with her for a while, and then decided to adventure with her. I liked her a lot as a character: she was a really good complement for my PC (I was kind of like a ranged DPS with my plasma rifles, she was kind of a tank with her power glove and eventual Brotherhood power armor). I also really liked her voice, and was trying to figure out whether I knew her from something else: was she an NPC in Dawnguard or something? After that play session was over, I went online and looked her up in the wiki. Oh, duh: Felicia Day! Like, one of my favorite people associated with video games! I felt a bit dumb to have not immediately recognized her voice, but really happy at the thought of traveling with her for longer than, say, Tallis.


Along the way, I started making friends with the NCR. I popped into Fort McCarran and began running missions for the snipers there. (This was also the site of a really awful mission, which takes one of the worst things that can happen to a woman and turns it into a side-quest, which has the least-sensitive dialogue I've encountered in any modern game.) By the time I finally entered the Strip, I was starting to see how the game was leading up to a showdown between the NCR and the Legion, and felt pretty comfortable aligning myself with the former.

MEGA SPOILERS

Of course, it turns out to be a bit more complicated than that. Mr. House has been preparing his own scheme, playing the NCR and the Legion off against one another, with the goal of maintaining Vegas's independence. And, while taking care of Benny, I stumbled across Yes Man, who seems to be a mechanism for charting a truly independent course, possibly with you setting yourself up as the King of the Wasteland.

What had initially seemed like a clear-cut choice had just grown much murkier. I tend to really enjoy playing good-hearted opportunistic rogues, and for a while tried to follow up on the Yes Man options, thinking that I could raise an army of my own and then join forces with the NCR. It turns out that this isn't allowed, though, as the NCR won't look too kindly on paramilitary activities within their territory.


Similarly, I'd held out hope for a while that I might be able to convince Mr. House to join with the NCR. There were a few other cases where the NCR had sent me to destroy a faction, like the Great Khans, and I eventually was able to work out a diplomatic solution. That also wasn't possible, though. Mr. House sees the NCR as corrupt upstarts, completely undeserving of his vast intelligence and preparations.

I held off for a while, but eventually my progression through the main NCR plot line forced me to make a choice, and I decided to stick with my original plan and take out Mr. House. I was pretty surprised by how guilty I ended up feeling! Based on the information you learn after returning the Platinum Chip, and in your Pip-Boy messages after the death, it seems like Mr. House was one of the best, and possibly the only, chance for humanity to repair itself after the devastation of the Great War. Mr. House had begun preparations even before the first bombs fell, and had the resources and ambitions to secure a reliable source of power for humanity, with the ultimate goal of restarting the space program and settling on an un-despoiled planet.

But, again, this is an interesting game with compelling and varied characters, and there's no clear-cut answer to the question of who is "good" or "bad." Mr. House had noble aims, and, based on what I saw, very little personal ego: he wasn't doing this for self-aggrandizement, but to fulfill a lifelong mission. And yet, he sits at the heart of a dictatorship, and that dictatorship would only spread after his plans were fulfilled. In contrast, the NCR is a messy republic, with the corruption, bureaucracy, and expansionist urges that implies. And yet, the NCR is a democracy, the only large-scale one of its kind: a flawed system, but a system where every citizen can enjoy its benefits and help shape its future. In essence, the question was whether to try and save the social structures of pre-War society, or to save its technological superiority. The NCR offers the first, and only Mr. House can provide the latter.

As usual, I can't help but think about what D&D alignments to assign each faction. Caesar's Legion is pretty obviously Lawful Evil. Caesar united the wasteland tribes, and as he describes their philosophy, it's one that completely based on subordinating the will of the individual to the military and the government as a whole, which extends to such matters as forbidding women to enter the military and maintaining a system of slavery seemingly for its own sake. Mr. House is Neutral Good. He has positive goals, but doesn't particularly care what methods he takes to achieve them. The NCR is... I actually have some trouble with this one. My immediate thought would be Lawful Good, due to their focus on government and bureaucracy and their role in opposing the Legion. But, when you hear stories about ranchers being forced off of their land, or incidents like the Bitter Springs massacre, it becomes harder to place them. I think they aspire to Lawful Good, but probably end up Lawful Neutral. Their overriding goal is stability and expansion, which accomplishes a lot of good along the way and certainly makes life safer for those within its borders, but often ends up causing collateral damage along the way. (I didn't get far enough along with Yes Man to categorize that. It definitely seems Chaotic, and I suppose its exact nature might depend on who you bring in to your alliance. Allying with the Omertas and Boomers might be Chaotic Evil, while the Brotherhood might allow you to be Neutral or Good.)

Anyways! After committing myself to the NCR's cause, I had a very brief period of detante with Caesar. At the time I was still hoping to seize control of House's Securitrons and give them to the NCR, so I entered the Legion camp in the plans of upgrading his robots. This turned into a more complex situation, and I ended up playing along with Caesar's quests for a short while. It was pretty fascinating to talk with him: based on everything I'd heard about him up until that point (granted, primarily from the NCR), I had him pegged for a straight-up villain. And, certainly, you get plenty of direct evidence of that when you enter camp: you see people crucified up on crosses, and slaves, and all sorts of awful stuff. But, Caesar himself is surprisingly erudite. It turns out that he actually was born in the NCR, and was a member of the Followers of the Apocalypse (a pacifist order devoted to improving peoples' lives through rediscovered technology) for most of his early life. Along the way he read Hegel, and began to examine the NCR dialectically. He became convinced of the inevitability of the NCR's fall, and decided to take the part of its conqueror. He united the tribes using brutal but cunning means, and has amassed a world-changing amount of power. What's interesting to me is how detached Caesar seems from what he's created. He sees history as a set of immutable principles, and doesn't feel he deserves much personal praise or blame for the part he plays in it.

I'm glad I took the time to chat with him and figure out what made him tick, but it also reassured me that I was making the right choice in backing the NCR. (I'm pretty picky about morality in games: I hate it when it's too black-and-white, but also get annoyed when everything is treated as equivalent, when it really isn't.) I upgraded the robots, killed Mr. House, felt bad for a bit, uploaded Yes Man, then realized that I wouldn't be able to do anything more with him.

From here I hurtled forward into the endgame. Unlike the smaller factions, which usually center around one or two questgivers in a single location, the NCR progresses through a wide range of ranks and areas, from the McCarran airfield to the McCarran terminal to the Strip to Hoover Dam itself. I really liked the various people I came into contact with along the way, and particularly liked Colonel Cassandra Moore. She fits nicely into the role of "badass female military officer" that I've been missing since Commander Shepard departed. (And, while I'm certain that the name is a coincidence, it also augers nicely for another military Cassandra who should be gracing our hard drives this fall.)

While Col. Moore was a great character, her quests were also the most trying for me, as she has little patience for diplomacy and pushes hard for results that change the facts on the ground. She'd gotten irritated earlier at my solution to the Great Khans problem; while the Khans and NCR are mortal enemies, I'd been trying to maintain good relations between them ever since peacefully resolving a hostage crisis, while she would have preferred to see them simply eliminated. Similarly, she sent me out to eliminate the Brotherhood. This was an especially tough row to hoe: I'd grown acquainted with them while completing Veronica's personal quest, and based on what I'd seen, they didn't seem like any sort of threat to the NCR, and I imagined they'd continue hunkering down below the earth until they all died out. But, they had also been incredibly cruel to Veronica, so my sympathy was somewhat limited.

I'm really glad that I saved my game before starting this quest, because it was one of the few times that I was unhappy with how the story was progressing and decided to rewind. I had a hunch that it would be possible to find a peaceful solution, and I hated the idea of murdering all the people I'd been chatting with just a few days earlier. I talked with Elder McNamara, and warned him about the NCR's plans; he gently dismissed my concerns. I next thought that Paladin Hardin might be more receptive; he seemed more aggressive and less open to diplomacy than McNamara, but I figured that if I set him on the throne, he might be willing to listen to me. So I did his side-quest, and ended up with him in charge, but still no peaceful option available. I gave in and looked online, and found that I would need to gain a positive reputation with the Brotherhood while keeping McNamara in charge; so I reloaded and did so. (Pro tip: you only get one chance to use your dialogue option warning McNamara about the NCR, so be sure not to use it until you are at least Accepted by them.)

Moore was not happy about this - she didn't trust them, and complained about how the bureaucrats would swarm all over this - and it was one of the only times that my NCR reputation actually dipped. Totally worth it, though! Once again, I was happy to see the game present me with a challenge that didn't have a cut-and-dried solution. What I ended up doing was much more difficult than just carrying out the mission would have been, and didn't give as many in-game benefits, but felt all the more rewarding due to its difficulty.

Along the same lines, Obsidian did a pretty brilliant job with a very minor side quest that I stumbled across while going to visit Caesar. You run into an NCR checkpoint; due to my high Reputation with them, I was able to talk with the commanding officer about the situation. Some of his men had been captured by the Legion and put up on crosses; from prior encounters with the enemy, he knew that if he ordered his men into battle, the Legion would torture and kill their comrades as the attack commenced, which would destroy morale. Since I was still a bit of an outsider, he wanted me to help him do the difficult work of giving his men quick and merciful deaths, to clear the way for the attack.

Well. I'd been willing to do some challenging things during the game, not least among them killing Mr. House, but I wasn't about to add killing prisoners to my roster. So, even as addicted as I am to side-quests, I turned him down. There weren't any dialogue options for alternate solutions, and no quest entries or notes describing other ways to finish it. But I still felt personally compelled to do what I could to rescue these men. So, I had Arcade Gannon and Rex stay behind, and snuck forward into the town. I sniped a few patrolling guards when they got too close, then spotted the gallows where the NCR troops hung. I crept forward, boarded the platform, and quickly cut them down. The alarm was quickly raised, and Legionnaires poured into the square. I drew their fire, cutting down the sniper units and evading the melee fighters, buying my comrades the chance to escape.

I'd wondered if this was an unintended glitch or something; open world games like this often support unplanned solutions to problems. But, when I escaped and returned to the checkpoint, I found the commanding officers in high, chagrined spirits. He admitted that he hadn't thought what I was done was possible, and thanked me for saving his men.

Well! It's a bit hard to convey how radical this is, but when you have a game that's entirely built around sidequests, and every sidequest gets an entry in the DATA/QUESTS tab of your Pip-Boy, it's extremely easy to fall into the habit of thinking that there are no solutions available other than those which are explicitly mentioned. This was a case where the game gave absolutely no hints to believe that an alternate outcome was possible; I felt compelled to follow my own moral compass and ignore the instructions of the game; and then was delighted to discover that it had essentially been a test all along, and I had passed. I've rarely felt so good after finishing a side-quest in a game.

There are several big, impressive setpieces that wrap up the game, or at least the NCR plot path. There's a nice, complex sequence when President Kimball arrives at Hoover Dam for a public relations junket, decorating a soldier and delivering a speech that will express the NCR's positive intentions for the region. Of course, as all of the soldiers expect, the Legion takes advantage of the situation to plot his assassination. This was a really fun and pretty tense mission: you have to do some legwork to try and anticipate the nature of their plot, position yourself appropriately to monitor what's happening, then detect and quickly respond to multiple threats. The speech ends up being cut short and the President evacuates, but your star rises even higher.

After this, the game clearly warns that you are entering the "point of no return": the final mission that will end the game. That was a bit surprising! In all of the other Bethesda games I have played, there might be one or more big "final missions" that wrap up a major plotline, but it's still a big open world and you can keep playing it for as long as you want after "beating" the game. In contrast, New Vegas has a very clear climax, whose outcome will forever change the future of the Wasteland. I had initially thought I would approach this like I do most non-BioWare RPGs these days: do all of the major storylines and as many sidequests as I could stand, then, once I feel like my interest is waning, do the final mission and beat the game. Now that I was actually staring at the screen asking me whether to proceed, though, I came to a realization: I didn't want to do everything. I wanted to do this thing, to finish the path this character was walking along. Not because I was sick of the game: the opposite, actually. This game was so vast, and so varied, and so good, that I knew I would be replaying it in the future, and wanted to save some of the missions for future runs.

The very final mission is, of course, the Second Battle of Hoover Dam. It's a long and nicely challenging quest. There are a ton of Legionnaires invading the Dam, and my beloved Lightweight Leather Armor went from full repair to dangerously low condition before I had even left the dam building. Fortunately, the NCR rangers are fighting by your side, and the firefights ended up being much more complex and interesting than I was expecting. Often I would end up pinned in the middle, between NCR sharpshooters on the upper levels and Legion forces racing upward from the turbine intakes. ED-E helped take out enemies from far away, and Veronica managed to keep most of the bad guys off of my body while I focused on headshots with my Q-35 Matter Modulator.

My one complaint about this sequence was that the Dam is huge and difficult to navigate, and for once the direction indicators on your Local Map aren't very helpful. I wandered around for a long time, getting ambushed again and again, went outside, went back inside, and finally had to look online to find out where I was supposed to go to turn on the turbines. Once that was done, though, the endgame moved along at a really nice clip. There's a long way to travel along the dam, with lots of ambushes and counter-ambushes. (One very minor complaint: I was kind of expecting more support from my allies during this sequence, but it seemed to pretty much just be the NCR fighting on my side. I did see a plane drop a bomb on a bunker, though, so that was probably the Boomers? Still, I don't think that the actual gameplay would necessarily have improved with more boots on the ground, and the allies' fates were described well in the actual epilogue.)

The NCR fell behind once I entered the Legion camp, where resistance was surprisingly light. I ambushed three guards and moved forward, where I saw Legate Lanius kill two Rangers with his sword. He stepped forward to taunt me, and I gradually realized that - incredibly! - this game could be beaten without killing the final boss. It's the only case I can think of other than Planescape: Torment (which, again, many of the Obsidian team helped create) where a final confrontation can be resolved through either violence or words.

Mechanically, it's not that awesome - basically, you just need a Speech of 100 and to keep taking the Speech options in the dialogue. It would have been more interesting to have something closer to the dialogue mini-games of Mask of the Betrayer, where you need to read closely, grok out a character's opinions, and then carefully bend the conversation in that direction. But, the actual content is really interesting. Lanius expects you to argue that the Legion can't hope to beat the NCR, but you need to essentially argue the opposite. The Legion may beat the NCR, but after winning at the Dam, they will not face any opposition before the Pacific Ocean. They'll spread out over all that territory, growing incredibly thin: rather than a spearhead, they will become a cloak. Without the focus of an enemy to face, they will lose their reason for existence, their cohesion will fade, and they will lose their glory. It's the logical outcome of Caesar's beloved Hegelian dialectic. With enough loquaciousness, you can help Lanius see that a victory at the Dam would be even worse than a defeat, and he will accept that it would be better for the Legion to withdraw, keeping the NCR as an enemy without actually defeating them.

So, yeah... pretty awesome! After that's done, you leave camp, where you meet General Lee Oliver, who has led the troops marching in the opposite direction. There's some congratulations, and a very nice epilogue that covers a surprisingly large amount of afterstory, including the fates of various communities, companions, and factions. Not all of these were good for me, largely due to quests that I had never undertaken. On balance I was very happy with the ending, but I also have more incentive to go back and get a "perfect" outcome on another playthrough.

END SPOILERS

Like I mentioned above in spoilerland, I'm actually planning on diving back into New Vegas to replay as another character, aiming for one of the other endings. Not that I'm unhappy with the one I got, but there's so much left in the game that I haven't experienced, and I'm very curious to see what the other outcomes look like. That's become extremely rare for me: not counting Civilization, I think the most recent modern game that I've played more than once is the original Dragon Age: Origins, which came out about five years ago.

I did go ahead and make another photo album for this game. I've returned to my standard form for this one: I took absolutely no screenshots for the first half of the game, a scattered handful of shots after the halfway point, and way too many near the end of the game. Spoilers, etc.

Two big thumbs up from me for this game. It's probably come closer than any modern game to capturing my Platonic ideal of an open-world game with a meaningful and engaging story. It's rich and vast enough to support seemingly endless exploration, but the central conflicts are written so well that you feel compelled to pursue them. It's a bit sad to realize that we're so far out from another game of this sort. Rumors started floating around late last year that Fallout 4 has started development and will be set in Boston, but even if those rumors are true we are probably many, many years away from a solution. If they're true, I hope Bethesda will entrust Obsidian with the franchise again, or at least take a sheaf of pages from their storytelling playbook as they work on the next entry.

Wednesday, April 09, 2014

Peace and Quiet

I enjoy writing about games, but don’t really follow much games journalism. Most games writing is focused on upcoming games, and if it’s something I’m excited about, I generally want to avoid learning about it so I can directly experience it for myself; if it’s something I won’t enjoy playing, I’ll probably enjoy reading about it even less (unless the writer is particularly snarky). The one site I regularly follow is Gameological, a department within The AV Club pop-culture website, which focuses on longer-form pieces, has really talented contributors, and a small but fervent and talented community.

Gameological recently kicked off a new feature that will focus on the “empty spaces” in games. I highly recommend reading at least the intro to their first piece, which does a great job at explaining this aspect of video games that’s fairly unique. Unlike a movie, TV show, or music album, which moves ahead at a deliberate pace determined by its creator, many games give the player agency to determine where they will go, how they will spend their time, and what they will see. In a few rare, delightful cases, you can come to enjoy simply spending time in a corner of the world: not advancing the plot or deepening your understanding of another character, but finding a deep sense of restfulness that you can inhabit and share with your character for as long as you like.

Honestly, that’s one of the most memorable aspects of the Elder Scrolls games for me. A few years after I finish each game, I’ve forgotten the majority of the plot, but the sense of space and quiet continues to linger with me. I remember traveling through Cyrodil in Oblivion, seeing a mountain far off in the distance, and thinking, “Huh, that looks cool.” Then I hopped on my horse, and rode in real-time towards that mountain, watching it slowly grow closer and closer. I rode as far up the paths as I could, then dismounted and worked up the rocky crag to the very peak, where I could turn around and look down over the incredible valley below, including the spot where I had first spontaneously decided to embark on this journey. Not a quest: there was nothing important up here, no notes in my journal telling me to come here, no experience rewarded for making it all the way. The beauty of the surroundings was its own reward. It’s natural in these games to always rush ahead, trying to cut down on the To-Do list in your journal, but here I waited for many minutes in real-time, watching as the sun slowly finished its arc across the sky, and set in a glorious wash of purple and orange over the horizon, as Cyrodil quietly and gracefully embraced twilight.




Empty spaces have grown bigger and more detailed in recent years, but I’ve long enjoyed them. One of the earliest spaces I can remember that probably fits the bill is Erana’s Peace from the original Hero’s Quest (later Quest for Glory). Much of that game felt very dangerous: the town was safe, but whenever you ventured into the woods you risked being ambushed by goblins, or brigands, or worse. A few unique locations in the woods helped advance the plot and introduced new characters, but usually held their own hazards as well. The game also featured a day/night cycle, and if you didn’t pay close attention to the light, you could get into trouble: the town gates shut after the sun sets, and if you get tired and try to sleep outside the safety of town, the night gaunts will get ya and you’ll immediately die.

Erana’s Peace was notable as the single spot outside of town where you could safely sleep. Aside from this, it didn’t have much connection to the game, but there was a mystique around it that fascinated me. You hear a few whispered stories about Erana, and the good she did for the valley, but while you continue to benefit from her grace you know so little about her. Perhaps most importantly, Erana’s Peace had a unique theme, composed by the terrific Mark Seibert, that I never, ever got tired of hearing. Even when I didn’t need to worry about the night gaunts, if I was in the neighborhood I would always try to drop by Erana’s Peace, where I could eat fruit from the tree, sleep on the grass, and enjoy the music and beauty of the valley. Eventually I would need to leave and challenge the evils confronting the valley, but I treasured the time I could spend there.




Some other empty spaces that come to mind include:
  • Practically all of Britannia in the Ultima series. Even more so than the Elder Scrolls games, Ultima was a fantastic space for free-form exploration and roaming, or just chilling out. You could spend a day harvesting wheat in the fields that you would later bake into bread; or sit and watch a play being performed about the exploits of the Avatar; or sit in the tavern in Jhelom and enjoy the good-natured brawl that breaks out every night; or climb into a hot air balloon and just let the wind blow you around. Most of the land was just wilderness, though, and I found walking through Britannia an oddly meditative experience, to the point where I tended to resist using moonstones except in times of danger.
  • Moving to the real world, the modern Grand Theft Auto games have shared a similar dedication to world-building, and each of their cities are crammed with interesting, engaging spaces that are totally disconnected from the plot. Mount Chiliad is a HUGE location, and could take hours to fully explore, but it’s a lot of fun to just stand at the top and gaze down at all the lands below. Similarly, I remember being shocked at the end of GTA IV where you end up on their equivalent of Governor's Island: I hadn’t realized that it even existed, and had never gone there before, so I ended up walking around in bemusement, watching the crowds of tourists chattering as they explored it by my side. After the heaviness of the game preceding it, it was a surprisingly welcome respite.
  • And, projecting into the future, the Fallout games have done some terrific work with the eerie, deserted atmosphere of a post-apocalyptic wasteland. Everywhere you go, you see the relics of a much busier time: freeway overpasses, now torn in half and gaping over a river; road-side tourist traps that haven't seen an automobile for over a century; enormous city libraries with every volume burned; and everywhere you go, discarded soda bottles, snack cakes, and cigarettes. The landscapes in Bethesda's Tamriel are empty because they haven't been developed yet, so there's a sense of optimism amidst the danger they pose. The landscapes in Bethesda's Fallout are empty because they were destroyed, and you are constantly reminded of the fact that they used to be teeming with life. This lends a constant melancholy to the series, and is one of the best examples of the dichotomous tension it builds between the vigorous, upbeat, cheerful optimism of the past and the grim, dark, hopeless reality of your present.

  • The house in Final Fantasy VII. VII was an RPG pioneer in many different ways: it introduced romances several years before BioWare’s pioneering BG2, and it’s also the earliest game I know of that introduced the idea of offering real-estate as an end-game money sink. Usually, late in an RPG, you’ve already bought or acquired the best weapons, spells, etc.; but the enemies you’re killing now are giving you more money than ever before, and it can be annoying to build up a large bank account with nothing to spend it on. So, clever games will give you one or more absurdly expensive options to buy a house or other property. There’s little or no in-game benefit to owning this, so players don’t feel compelled to grind for it; but it’s also unique and cool enough to feel worth saving up and spending money on. Anyways, in VII you could buy your own resort villa for the staggering price of 300,000 gil. Once purchased, you can walk in, sit down, put your feet up, and say, “This is MY house!” Useless, but oddly comforting. I would sometimes leave Cloud here resting while I took my own break to use the toilet.
  • ICO. ICO actually might be the best example of the sort of empty spaces Gameological is talking about: a vast castle, marked by an ominous history, but isolated and practically deserted. Much like the puzzles in an earlier generation of adventure games, you’re invited to explore and progress at your own pace, without flashing arrows popping up on the screen directing you to the solution. (Yorda might occasionally point at something she thinks is interesting, but is much more likely to play with the birds or stare at her reflection in the water.) It's also yet another beautiful space; even though your ultimate goal is to escape, it's easy to forget that while walking through the grass under dappled shade while a windmill creaks in the distance.
  • This probably doesn't count, but when I was younger, I would continue playing Civilization games after I had conquered the world. All my enemies vanquished, I had now united all of Earth's various tribes under my benign reign, and, freed from worry of backstabbing rivals, I could turn the game into a kind of large-scale version of Sim City: planning where I would plant my next outposts, how to grow my population, what buildings to construct, the amount of care to show for the natural world. But, unlike Sim City, the memory of conflict remained fresh in my mind, making me appreciate the calm planet much more than I would if I'd never faced any opponents from the start.
Of course, there are many more examples, and I'm looking forward to seeing what other ones Gameological comes up with in the months to come.

Thursday, April 03, 2014

Paperhouse

Wowza... just finished House of Leaves. It's definitely one of the strangest books I've read in a while. It's one of those books which is strange both in structure and in content. Reading it has been a bit of an expedition in and of itself: not unlike going on a hike, you go through peaks and valleys, learning how to navigate the different terrain you encounter, gradually getting to know the land better as you continue towards your destination.

I have a hard time comparing it to anything. It reminds me most of The Raw Shark Texts, though that's probably not a fair comparison, since TRST came out after this (and very well might be inspired by it). House of Leaves is darker and more bizarre; if pressed, I would probably categorize House of Leaves as an experimental literary horror novel, while The Raw Shark Texts is more like a literary thriller. However, the books share a similar creative use of typography, where the words aren't just used for the semantic meaning they carry, but arranged in particular ways on the page to help tell the story.

Here too, though, House of Leaves goes far beyond, with a truly dizzying number of techniques. The first and most obvious is its extensive footnoting: around 500 for the entire novel, ranging from a single word to a dozen pages in length. The footnotes come from three separate authors (or, rather, two authors and "The Editors"), with the later authors often referring to or correcting the earlier notes, rather like later sediment covering original stone. At first glance it seems like a work of academia, between the footnotes, appendices, exhibits, and index. However, as you delve deeper into the book, its strangeness becomes ever more apparent.

MINI SPOILERS

One of the high points for me was a chapter that I think of as "The Labyrinth" (and, in fact, receives that as a proposed name near the very end of the book). Narratively, at this point in the book the characters have started to explore a mysterious series of rooms and hallways that have impossibly appeared within their home, extending into places where they shouldn't exist. As they quickly learn, the architecture of this place is strangely mutable: doorways appear and disappear, rooms grow bigger or smaller, new turns appear in halls. Most of the men are excited to explore, but this is balanced against the danger of losing their way.

Appropriately enough, the text in this chapter forces you to experience the same disorienting relationship to changing space as the characters are experiencing. You read the text, which includes a footnote. You read that footnote, which refers forward to another footnote several pages later. You flip to that later footnote, and see that it links to two different footnotes, and must pick which of those to follow, each in turn leading down their own chain. Eventually you reach an end, and backtrack to the earlier junction, venturing down the other path. Then later on, you intersect with those paths you earlier crossed, and can either revisit them (entering the rabbit hole again) or press onwards. The whole experience feels like reading a Choose Your Own Adventure novel, except for its dry technical language and underlying dread.


In many instances, it becomes clear that it's the structure itself that's important and not the meaning: syntax rules, semantics holds no power. There are some extremely long lists, such as naming every architectural style that's ever existed, or thousands of photographers. I have to admit that I read through all of them, and felt a bit like a sucker doing so: there didn't seem to be any hidden meaning or message within them, and I don't know enough about the fields to tell if there's some deep Joycean stuff going on in there. (For example, pretty much the only photographer whose name I recognized was Robert Mapplethorpe, but I'm left wondering if he was a sole real name mixed within all the fake names invented for the book, or if those were all real photographers in the first place.) However, there's a certain power that accumulates from such persistence in lists, creating a sort of droning, monotonous, monomaniacal obsessive atmosphere. Beyond that, though, the lists became more important for their presentation within the text. In an early case, a long footnote was split across small sections of successive pages, leading you further down and down and down as you read; then, once you reach the "bottom", they flip around, leading you gradually back up, page by page, the way you came. In another case, a footnote "burrows through" the page, leading you down a small box embedded in the middle of the page; the reverse of the previous page can be viewed on each opposite page, as if the ink had burned through.

These modes grow more elaborate as you progress deeper into the labyrinth; at one point, I sat in front of a mirror, holding the book up beside me, so I could read the reversed letters making up one particularly nefarious footnote. I'm sure I looked like a crazy person doing that. I'm sure that was the point. Likewise, I was the guy on the subway who would hold up a book, looking closely at the page; then flip the book 180 degrees and read it upside-down for a while, all the while flipping back and forth as though looking for something. Just saying: if you see something, say something!

I don't think I've experienced anything quite like the labyrinth in another book. In a certain way, it reminded me a little of the Circe chapter of Ulysses, where the established and familiar narrative style drops away and leaves you in a different mode, feeding on the intensely distressing activities taking place. But, where Circe occurs near the end of Ulysses and serves as a kind of climax, here the Labyrinth comes near the start of the novel, and helps establish many of your expectations for how the remainder of it will work.

At long last, you emerge from the labyrinth, and here too the typography continues to make you experience the story through the mechanics of reading. At its height, the labyrinth was a dense clustered mess with dozens of separate footnotes, all in tiny type, filling every square millimeter of the page. Now, though, you escape the maze, and into the enormity of the Great Hall. Now a single page might contain only a single line of text. From page to page, the text may grow gradually lower, as the characters descend the Staircase; or it might cluster at the top of the page, as they need to keep their heads down to stay beneath the ceiling; or divide into two pieces as the characters split up. Later in the book, the text becomes entirely unmoored, as Navidson loses all sense of direction and gravity itself seems to cease.

While there's a spacial metaphor at work here, there's a temporal one as well. One of the biggest dangers of Navidson's house is that they can't know how long a journey will take. You might spend a day walking down the stairs, and then seven days climbing back up. That adds to the disorienting, sinister atmosphere within the house. Likewise, there was a point while I was reading The Labyrinth when I was sure I wouldn't finish this book before July: each page was so dense, and just had so much text, that I felt like I was proceeding at a crawl, and was well aware of how many hundreds of pages still lay ahead of me. But, conversely, I flew through later sections simply because of how few words there were; even taking my time to savor the sense of spaciousness, I could easily read several dozen pages in less time than a single page in the Labyrinth would. All that to say, at any given point I wasn't at all sure of how far along in the book I was, and couldn't get my bearings. Which made me think about something that I, for one, don't even think about when it comes to reading (physical) books: by their very nature, we know how far we are, how much is left to go, and what we should anticipate. If you're at page 450 in a 500 page book, you automatically (and, by this point, probably subconsciously) know that the climax is imminent, and are already preparing yourself for the coming drama and the wind-down of the story. In House of Leaves, though, I always felt adrift (in a good way), never sure of where I was or what to expect.

As for the story itself, it's very hard to read this book and not think of The Blair Witch Project, Paranormal Activity, or any of the other "found footage" horror movies of the last 15 years... which would be a shame, except that House of Leaves was started about a decade before Blair Witch kicked off the sub-genre, and at the very least anticipated it and at the most inspired it. The content of House of Leaves focuses on a film called The Navidson Record, which, like the later actual movies, was shot by "ordinary" people who didn't realize the troubles they were about to encounter. This film is discussed as a real, historic artifact, which has inspired a vast number of critical articles and previous academic papers examining its artistic and metaphysical implications. The book does a lot to ground the movie, describing how a teaser was released on bootleg VHS cassettes and gained interest in the underground festival circuit, then how the Weinstein Brothers eventually purchased the rights to the complete movie and released it through Miramax, along the way quoting reviews from Anthony Lane in The New Yorker, along with a very wide-ranging set of interviews with luminaries such as Stephen King, Anne Rice, Harold Bloom, Hunter S. Thompson, and more, all providing their own candid reactions to the events of the film.

Of course, this is all fictional; what's really cool about it, though, is that it's fictional within the context of the novel itself. This is one of the most heavily-mediated works of fiction I've encountered. The actual story of The Navidson Record was written within the novel by an elderly blind man named Zampano. (The irony of a blind man describing a movie is an irony present but not commented on throughout the whole novel.) Zampano dies, seemingly of natural causes at an old age, but there are some mysterious details surrounding his death that alert us to something strange afoot (he kept dozens of cats, who all disappeared shortly before his passing; the floors of his apartment have been marked with what seems to be the deep claw marks of an enormous beast). His finished but unedited manuscript is discovered by a stranger named Johnny Truant, a Los Angeleno tattoo assistant in his early twenties. Johnny takes the manuscript home, and eventually decides to organize and edit the many disparate pieces of the manuscript (handwritten and typed in a variety of different hands and styles). Along the way he inserts his own footnotes, which depict his increasing unease with the content of the manuscript, and a growing sense of horror as the terrors contained within the fictional novel seem to be intruding on his real life. Johnny grows increasingly unhinged as the novel progresses, starting to hallucinate and growing obsessed about The Navidson Record. He intellectually knows that it isn't real - no such movie was ever released, none of the real people described within the text ever said the things attributed to them, all the footnotes are either citing publications that don't exist or articles that were not published within them - but at the same time, he can't shake the conviction that the events which occurred within this work of fiction are reaching out to smother his non-fictional (from his perspective) existence.

And then, in a final layer of mediation, Johnny's manuscript itself finds its ways into the hands of Random House Books, the real-world publisher of the novel, and a final level of authorial gloss is applied over the entire book. This final level is less present and visible, mostly providing some missing translations from French and German, or explaining some of Johnny's stranger statements. But, of course, this final layer is also the most present, since it's presumably the author himself who is presumably inventing all the layers below him. (Once again, I often found myself thinking of the concept of The Wick from Neal Stephenson's Anathem here, as I shifted between "higher" and "lower" levels of reality or fiction.)

All of this was pretty fascinating on its own right, with each layer commenting on the reliability and faults of the layer below itself. It was also pretty interesting in the context of a horror novel, too. When you think about it, the job of any horror book or movie is to take a fictional fear from a work of art you're experiencing, and seek to convert that into a present fear that infects your actual life. So, you see a woman stabbed to death in the shower in Psycho, then feel a twinge of fear the next morning when you take a shower. Rationally, you know that there's no connection between the two, that there's a barrier which keeps the monsters from invading our lives... and yet, very well-constructed works of horror can weaken that barrier, letting us mentally superimpose the unthinkable onto our ordinary lives. I think that, by repeatedly showing how bad events from one acknowledged work of fiction can cross over and cause damage in a supposed life of non-fiction, House of Leaves primes our minds to mentally continue the chain: Zampano created the House of Leaves, and then was killed by the creature he invented; Johnny read Zampano's novel, and then was pursued by that same beast as it leaped into his world; and now we are reading Johnny's about desperate position. What will happen to us? (Again, to cite the horror movie trends of the last fifteen years, it's the same general idea shown in Ringu/The Ring. We've already seen how a movie can violate the laws of movies and break into the real world. So what happens to us now that we've seen that same movie?)

Johnny's unraveling is probably the most disturbing aspect of the book. He seems like a bright, charismatic person, albeit one with a troubled past. As he grows more obsessed with The Navidson Record, he begins to encounter truly macabre scenes: often accompanied by horrible stenches of death and decay, he witnesses catastrophic accidents, suffers grievous injuries, sees predators approaching him, losing control of his bowels. And then, we immediately learn, none of it actually happened. The car didn't crash; his arm isn't broken; there's no creature there; he hasn't soiled himself. In almost any other book or movie, this would feel like cheating - "Oh, it was all just a dream!", a way to get the cheap thrill of a jump scare without responsibility of following up on an actual plot development. But, within the particular context of House of Leaves, this very same technique is actually extremely effective. The very threat that Johnny is facing is a disintegration of his mind, as all those aspects of him we liked are slowly drawn away from him and he is turned into the husk of a man. Or, what if they aren't hallucinations, and he isn't losing sanity? Well, that's even worse, since that means that the evil is invading this world in very real form.

All this talk of "horror" and "darkness" is very vague, and you might think that I'm being unspecific to avoid spoilers, but that's only a small part of it. The very nature of the badness in this novel remains mysterious throughout, simultaneously impossible to deny and impossible to comprehend. In horror, it feels like there are two different types of approaches to take, the very specific and the very general. Specific horror gives us a very detailed and highly realized threat to fear: Norman Bates, The Gentlemen, Freddy Kreuger. General horror provides a lingering atmosphere of dread and free-floating anxiety, denying us the release of confronting our foe: The Haunting, H. P. Lovecraft, Tristero. House of Leaves falls very decisively into the general camp. There's some speculation as to the nature of the threat they face, and it certainly seems to be very old and very powerful, but any time they seem close to meeting it, it's revealed to be something else. This lets the fear grow bigger and bigger as the novel progresses, and, as is usually the case, ultimately ends up being far more frightening than a specifically-identified foe would be.

 So, Johnny is definitely an unreliable narrator, and Zampano even more so, particularly since he's an author creating a work of fiction within this world. It's interesting that, while Johnny is supposedly the one who is performing the non-fictional work of fact-checking and organizing Zampano's text, his passages are the most novelistic, with vividly drawn characters and lots of action. In contrast, Zampano uses a very dry academic tone throughout, but he's ultimately describing fiction. It's a fascinating job Zampano's set himself to, inventing hundreds or thousands of straw men from other academic papers that he has made up, and then taking the role of an impartial observer, being careful not to offer his own opinions of this fictional movie (which he created), but citing independent sources to offer balanced opinions about it. In the few instances where he drifts from this mediated style of writing, though, Zampano seems to let some of his own (potentially real-life) prejudices slip through. For example, Zampano seems to have a very negative view of Karen Green, Will Navidson's partner. He regularly implies that she is unfaithful to Will, and a bad mother to their children. However, whenever Zampano gets around to actually sharing the details (hundreds of pages after his first insinuating remarks), the evidence seems remarkably flimsy. After talking about how Karen "cheated" on Navidson with Wax, we "see" the footage that caught the act, which showed Wax kissing her in the kitchen, then her reacting in surprise, and then leaving the room. Not exactly the act of an adulterer. Later on, Zampano seems smugly righteous when he writes about Karen's affair with an actor; but his only sources for this infidelity are the actor himself and Karen's sister, both of which are presented within the text as being supremely untrustworthy. Anyways, it's interesting to think about what, if anything, we're meant to draw from this. Is Zampano (the heavily-mediated author) subconsciously projecting his own misogynistic prejudices down onto the characters he created? Is he deliberately crafting these people to play roles, and just doing a poor job of establishing them? Or is something else going on?

I also thought it was interesting that Zampano's attitude towards Karen seemed to improve so much after she left the house. He starts giving her more of the benefit of the doubt, and acknowledging the stresses she's been under, not just during the explorations but for her entire life. In the end, there's actually a pretty heartwarming reunion between the two, ending things on a surprisingly high note. That, in turn, made me wonder if Zampano's insinuations were crafted in order to give her more of a dramatic arc: an invented redemption might be better than no redemption at all.

Finally, a few thoughts on the title. The word "house" is very prominently featured throughout the book, and at first it seems obvious that the book is named after the Navidson house. That said, where does the "of leaves" come from? Leaves really aren't mentioned in any capacity throughout nearly the entire book. After I'd read more than halfway through the novel, I started to speculate on what the title could mean. My first thought, especially given the aforementioned collaboration between text-as-writing and text-as-meaning, was that the "of leaves" referred to leaves of papers, and therefore "House of Leaves" was... well, was "House of Leaves", the very book we were reading, referring back to itself.  The house is the book, and the book is the house. Like I noted above, many of the properties we ascribe to the house are shared by the book itself: the way it changes form, the way its passages divide and multiply, the way you can lose your sense of time and place within it, the way it speeds up and slows down, how you start to feel disoriented as the distances change, etc.

So, that was my working theory for a while, but then, later in the story, we see an actual book called House of Leaves. This is a book that Navidson has at a time when he is lost, and alone, and convinced that he might die. He's wanted to read it for a while; now he has the time, but he's nearly out of light, with only a couple of matches yet. He has several hundred pages of the book left, and only a few matches, each of which will only burn for less than a minute. And so, he comes up with an absolutely brilliant solution. He lights a single match, and uses its light to read a single page. Then, he rips that page out of the book, rolls it into a tight cylinder, lights it with the last stub of the match, and then uses the light of the burning page to read the next page. Each page takes almost exactly as long to read as to burn, and so he continues, racing to keep ahead of the flame, chain-smoking his way through an entire novel.

I don't know what that means, but I thought it was awesome, one of my absolute favorite passages in a book crammed full of memorable scenes. Interestingly, we never learn what Navidson's book House of Leaves was about: I'm very tempted to think that it's the same book we're reading, and Navidson is reading about himself, but I don't think there's anything in the text to support that, and it seems likely that Zampano would have mentioned if that were the case. More broadly, that scene seemed like such a great way to summon up the mood of the book: hopeless, and desperate, but still able to appreciate beauty.

After the very end, in some of the end material that follows the conclusion of the main narrative, there's a poem (page 563 in my edition) that seems to obliquely reference the story we've read about, with lines like "walls keep shifting." However, in this context, it seems to be comparing a house of leaves to the world. When you think about it, whether they're leaves of paper or leaves of a tree, a house made of leaves is extraordinarily fragile. Any and every action would risk destroying it. It's funny that, in all the time I spent thinking about this book and the meaning of the title, I had completely forgotten that very literal implication of the name.

END SPOILERS

I'd like to end with a quotation that, while not representative of the story itself or its style, is a pretty great example of the stylistic play and structural ambitions of the book:

(Now that I think about it, I guess I've always gravitated towards written legacies (private lands surrounded by great bewildering oceans (a description I don't entirely understand even as I write it down now (though the sense of adventure about words (that little "l" making so little difference), appeals to me-- ah but to hell with the closing parent)he)see)s) (sic)

So, uh... yeah! I was going to say that this book left me speechless; as you can see from the mountain of text above, that isn't actually true, but it's certainly one of the most original things I've read. It would be an interesting, challenging read if only for its technical structure, but beyond that, it manages to weave a fascinating, odd, and occasionally terrifying story that stretches across multiple levels of reality. I can't say I recommend it to everyone, but if you enjoy reading strange fiction, or have just wondered what a horror novel written by a demented post-graduate student would look like, you should definitely pick this up.