(:htoc:)

Example one: Finder windows

Note for non-Mac-users: the Finder is the graphical file manager on the Mac OS. (It also does other things, but that's the part of its role that we're concerned with here.)

In all of the examples in this section, the data is the same: “what files are in this folder?” Let’s look at some of the possible views onto this kind of data.

Figures 1–4 show the same folder in each of the four different available view modes: list, icon, column, and cover flow.

Figure 1. Finder window in list view. Miscellaneous files.

Figure 2. Finder window in icon view. Miscellaneous files.

Let's play a game of “spot the difference” between Fig. 1 (list view) and Fig. 2 (icon view). Here we're not concerned with visual differences, but with UX differences. Here's a partial list:

(1) List view shows more metadata. (Here we see modification date, size, type; view options allow us to show more/other columns, like date added, version, tags, etc.)

Does the icon view show no metadata at all? Nope, it shows at least one piece of metadata: the file type—via the file icon. (This is an example of multiplexing; the icon has to be something—to provide a visual representation of a file, and to provide a click target—so why not multiplex file-type data into it?)

Of course, the icon is also visible in list view (but smaller); this means that in list view, file type is conveyed twice (if the “Kind” column is enabled). This is an example of redundancy in UI design, and of good use of sensory (in this case, visual) bandwidth (of which there is quite a lot!). Notice that this redundancy affords the UI a degree of freedom it would not otherwise have: the “Kind” column can be turned off (making room for other data columns, or allowing the window to be made smaller, to make room for other stuff on the screen) with minimal loss of information throughput for the UI.

But wait! What about the file name? There's metadata lurking there, too—the file type again, encoded this time in the file extension. Redundancy again; the file type is therefore displayed in three ways in list view (“Kind” column, icon, file extension) and in two ways in icon view (icon, file extension).

All of this gives the UI several degrees of freedom. How is this freedom spent? In at least two ways:

1. To allow for one or more of the channels through which file type information is communicated to be disabled or repurposed in certain circumstances, with minimal loss of information. (An example of a disabled channel: the “Kind” column is absent in icon view, but file type information is still visible. For an example of a repurposed channel, see the notes on Figures 5–9, below.)
2. To compensate for unreliability of some or all of the channels through which file type information is communicated. Sources of unreliability include:
   • The Finder may not recognize some obscure file types (the “Kind” column would then display no useful information); the file extension may be the only source of file type data in this case
   • The file extension may be missing (but Finder attributes may be set, thus allowing an appropriate icon to be shown and an appropriate value to be displayed in the “Kind” column)
   • The file icon channel may be repurposed (Again, see the notes on Figures 5–9, below, for an example)

(2) List view allows sorting. (Click on a column name to sort by that column's value; click again to reverse the sort order.)

… or is this really a difference? Actually, files can be sorted in icon view as well (there is both a “one-time sort” option and a “keep sorted by…” option). This is not obvious, because the UI for sorting in icon view is not discoverable by mere physical inspection, whereas in list view the column headers are visible, the sort order indicator is visible (the triangle, pointing up or down), and the “click column header to sort tabular data by that column's value” is a well-known idiom. (In icon view, sorting is done via a menu—either from the menubar, or from the context menu, or from a menu in the window toolbar.)

There is, however, a more subtle difference: in icon view it is not possible to sort in reverse order. Why not? The only reason is that Apple was unable (or unmotivated) to design a good UI for reversing sort order in icon view.

General lessons:

• The same (or analogous) forms of interaction with the data may be implemented via different UI designs in one view vs. another view.
• If the UX for a particular interaction in one view is obvious, don't assume that in other views it's impossible to design and implement.
• However, not all interactions that are possible in one view need to be (or can be) available in all views. (It makes little sense to provide “one-time sort” functionality in list view.)

Design principles:

1. In each view, provide as many interactions as is reasonable, no more and no less. (Provide more and you clutter and complicate the UI; provide fewer and some or all of the views will be too capability-poor to be useful.)
2. Strive to have each view provide as complete a set of interactions with the data as possible.
3. To reconcile the tension between the above two design principles, remember that it's better to provide a capability and hide it away behind a non-obvious or non-trivial-to-discover UI than not to provide it at all. This way, it will be available for power users but will not trouble less experienced users. (Of course, this is not an excuse to hide capabilities behind non-obvious UIs when there's a good reason, and a good way, to provide an easily-discoverable UI for them.)
4. At the same time, look for ways to exploit the unique properties of each view to provide additional interactions that would impossible or nonsensical in other views.
5. The more ways the user can interact with the data, the better.

(3) Icon view allows arbitrary grouping and arrangement; I can position the files in the window wherever I like (example 1, example 2, example 3, example 4, example 5).

(Unless a “keep sorted by…” option is enabled.)

Some file managers don't have this feature; the Finder does. The lesson:

Do not carry over UX/interaction limitations necessitated by one view, to another view where they are not necessary.

Arbitrary grouping and arrangement makes little sense in list view. In icon view, there's no reason not to permit it—except, of course, that allowing the user to set, and then tracking, arbitrary icon positions, takes work! Does it offer a benefit? Find out! Ask users, survey other implementations, etc. In general, users resent limitations on their freedom, and appreciate the lack of them.

(4) What aspect(s) of the data may be easily gleaned via visual inspection differs from one view to another.

Different views (usually) look different. It's easy to forget this, but it's crucial. Here (in the “Finder list view vs. Finder icon view” example) this manifests in a couple of ways:

1. In list view, it's easier to pick out files which differ from the others in any displayed metadata value (modification date, file name, etc.). This is true not only due to the sort feature, but also because humans find it easy to scan down a list of text items (which are horizontally aligned) and notice ones which stand out.
2. In icon view, the "file icon" data channel is wider (because the icon is displayed at a larger size); more data is coming through this channel. This makes it easier to distinguish icons, but also allows this channel to be used for other purposes (see notes on Figures 5–9, below).

General lessons:

For humans, the visual channel is a high-bandwidth one. Use it. Some ways to optimize UI visual bandwidth:

• Multiplex meaning.
• Allow the repurposing of high-bandwidth components.
• Remove obstacles to visual apprehension of patterns (minimize "non-data ink", etc.).
• Assist the brain's pattern-recognition abilities by using alignment, contrast, repetition, and proximity cues.

The same folder as in Fig. 1 and Fig. 2, but now in column view (Fig. 3) and cover flow view (Fig. 4):

Figure 3. Finder window in column view. Miscellaneous files.

Figure 4. Finder window in cover flow view. Miscellaneous files.

Figure 5. Finder window in icon view. Folder containing low-resolution icons.

Figure 6. Finder window in list view. Folder containing low-resolution icons.

Figure 7. Finder window in list view. Folder containing high-resolution icons.

Figure 8. Finder window in icon view. Folder containing high-resolution icons.

Figure 9. Finder window in icon view, zoomed to cover most of desktop. Folder containing high-resolution icons.

Figure 10. Finder window in list view. Miscellaneous files. No toolbar.

Figure 11. Finder window in list view. Miscellaneous files. One folder expanded to depth 1.

Figure 12. Finder window in list view. Miscellaneous files. One folder fully expanded.

Example two: Microsoft Word document windows

Figure 13. Word document window in draft view.

Figure 14. Word document window in print layout view.

Example three: structured data

1. A .csv file, displayed as plain text
2. The same .csv file, opened in Excel
3. An HTML file, containing the same data, plus markup such that the data will be displayed in tabular form, displayed as plain text
4. The same HTML file, rendered in a browser

(Analysis of examples two and three left as exercise for the reader.)

Of any skill, or any domain where expertise of execution may be gained, there are three levels of mastery.

At the zeroth level, you break the rules, because you do not know the rules. Success is accidental; failure is likely; excellence, all but impossible.

At the first level, you know the rules, and follow them. You do well enough, though excellence is unlikely.

At the second level, you know, not just the rules, but the motivations behind them; you understand why the rules must be as they are. You follow the rules or break them, as the task demands; your actions are governed by deep principles. Success is near-effortless; excellence becomes possible, and even likely.

To achieve greater mastery, you cannot skip levels. At the zeroth level, you may look at one who has achieved the second level of mastery, and note that he routinely breaks the very rules he has instructed you to follow. Are there no rules, then? But there are; and they exist for good reasons. You will not achieve the second level of mastery before the first.

Likewise, the one who has achieved the second level of mastery says to him who has yet to achieve the first: “Do as I say, not as I do”. This is not hypocrisy. One who does not understand the three levels may think: “He is allowed to break the rules, as I am not, because of some privilege of rank”. But it is only that to think outside the box, you must know the shape of the box, its contours; if you cannot see the box, you will never escape it.

And once more: you cannot explore the space of possibilities, if you do not know its dimensions. The axes of that space are not the bars of a cage, but signposts; not seeing them, you are not infinitely free—but only doomed to wander forever in a Flatland of amorphous mediocrity.

In appearance, it looks something like if Baskerville, a 225-year-old typeface that has been shown to shape our perception of truth, and Caecilia made a baby.

The Kindle Finally Gets Typography That Doesn’t Suck

These fonts are sort of like transitional serifs, but they’re also sort of like slab serifs, and sometimes they’re called “transitional serif but with features of a slab serif”² etc. etc.

… a crisp, modern serif typeface … avoids the stuffiness of historical text faces and doesn’t overreach when it comes to contemporary detailing … a balanced, low-contrast typeface with economic proportions…

Elena font description

Fonts in this category share these properties:

• fairly thick strokes in the normal weight
• low stroke weight variation
• serifs that are not sharply tapering nor thin and dainty, but thick (yet not geometric or square, as slab serifs)
• relatively open counters
• relatively large x-heights

… simply a contemporary body text font.

Tuna font description

Fonts in this category include:

• Charter
• Source Serif Pro
• Equity
• Chaparral Pro

… and quite a few more more—see the full list (that I’ve found so far) on my wiki (and feel free to suggest additions on the Talk page!).

Some samples:

The category does not seem to have any accepted name³—yet unquestionably this is a real cluster in font-space. This blog post is meant to call attention to the cluster’s existence.

The characteristics listed above mean that fonts like this will render well across a variety of environments, software and hardware. And empirically, these fonts make for pleasing and readable body text on the web. So, at least for now (unless and until someone tells me that there’s already an accepted name), I’m calling these fonts “screen serif” fonts.

If you want your pages to be readable and attractive, try setting your body text in one of these fonts! (Again, check out my wiki for the full list—I’ll be adding more “screen serif” fonts to it as I come across them.)

OK, it actually had other things in it, not just elevators. But the elevators were the heart of it—they were the most engaging part of the gameplay, with the most complex game mechanics—and, more than anything else, it was mastery of the elevator design that would bring a player success in SimTower.

I played SimTower a lot when I was younger. I never did manage to build a tower that reached the coveted “Tower” rating (the one beyond even “five stars”), but I spent many hours optimizing the elevators in my virtual office towers—and watching the tiny silhouettes of simulated people queue up for the next animated elevator car.

Today, in honor of that wonderful game, I give you a little puzzle of “elevator experience design”.

Many elevators have two call buttons: an UP button and a DOWN button. If the elevator car is currently at rest (for simplicity, we’re going to assume that there’s only a single car), then pressing either button will summon the car to your floor.

However—in a well-designed elevator system—the behavior of the buttons changes when the car is already in motion. If the car is currently moving toward your floor—having been summoned on a floor that’s past yours—then pressing one of the call buttons will cause the car to stop on your floor only if you press the button corresponding to the direction in which the car is moving. If you press the other button, the car will first proceed to wherever it’s going—moving right past your floor on its way there—and only then, reversing direction, will it proceed to your floor.

(An example: suppose the car is currently at ground level; it then gets summoned by floor 9. However, right as it’s passing floor 3, you—located on floor 5—press the UP button. The car will, then, stop at your floor (5) before proceeding onward to floor 9 and/or whatever floor you select. But if you had pressed DOWN instead, then the car would’ve sped past floor 5, stopped at 9, and only then considered your summon.)

So far, so good. Now we come to the puzzle:

Suppose that, instead of two buttons—one UP, one DOWN—there’s actually only one button. Should this button act like an UP button or like a DOWN button?

Come up with your own answer before scrolling down…

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

The first part of the answer is that it’s a bit of a trick question.

The question implies that there are only two possibilities for the button’s behavior. But of course there’s at least one other possibility (leaving aside bizarre designs, such as ones that involve randomization, etc.): that the button combines the behavior of an UP button and a DOWN button—in other words, that the single call button causes the elevator car to stop on your floor whenever it’s going to pass your floor, regardless of the direction in which the car is currently heading. (Let’s call a call button with this sort of behavior a “BOTH” button.)

So there are three possibilities for a single call button’s behavior: UP, DOWN, and BOTH.

You might suspect, now, that the answer I’m going to give is “BOTH”. That’s the point of the trick question, isn’t it? The possibility we didn’t think of, is the correct answer. This is a lesson about being careful in considering all possible options. Right?

Wrong.

The correct answer is DOWN.

Why is this? I’ll reveal the answer in the next post, and give my readers a chance to think about it in the meantime.

This is a shared interest.

(:htoc:)

Shared interests

A shared interest—in the way I am using the term—is nothing more than what it sounds like: an interest (in the broad sense of the word) that happens, for whatever reason, to be shared among all members of a group.

The distinction I want to draw is between a shared interest (of a group) and a collective interest (of a group). The former is a superset of the latter; all collective interests are, by definition, shared interests; but not all shared interests are collective interests.

Collective interests

What is a collective interest?

Well, suppose that I found another student organization (extracurricular activities look great on a résumé). This one is a Star Trek fan club. At the first meeting of Campus Trekkies United, we discover, to no one’s surprise, that all fifteen of us are fans of Star Trek.

… well, of course we’re all fans of Star Trek. That’s why we’re in the fan club in the first place! Anyone who’s not a fan, has no reason to join the fan club. And so: Star Trek fandom is a collective interest of Campus Trekkies United.

A collective interest is an interest that is shared by every member of a group in virtue of being a member of that group. Anyone who does not share that interest, will not be a group member.¹ And thus, by modus tollens: anyone who is a member of the group, will share that interest. It is guaranteed that every member of the group will share that interest.

Details & implications

Several important consequences follow from this.

Preservation of interests

Unlike a collective interest, a shared interest is not at all guaranteed to stay shared among all group members. Nothing stops someone from joining the Students Against a Democratic Society, who does not like Star Trek. At that point, Star Trek fandom ceases to be a shared interest of SADS. (Which may lead to some awkward consequences if, for instance, we had decided to start wearing colorful jumpsuits to our political rallies.)

Infiltration

I said earlier that “[i]t is guaranteed that every member of the group will share [a collective] interest”. But is this really true? Well, it’s true if the condition for an interest being a collective one holds: that anyone who does not share the interest, will not join the group.

But it is dangerous to simply assume that this condition holds, in the absence of any mechanism by which it is ensured to hold! Is Campus Trekkies United actually making sure that non-Trekkies do not join? Certainly it seems like they have no reason to want to join, but is that sufficient to keep them out?

Suppose a fan of Star Wars, incensed at the idea that the university would grant meeting space and funds to fans of the rival franchise, decides to pose as a Trekkie, and signs up for Campus Trekkies United under false pretenses. He hates Star Trek, and wants nothing more than to see the club cease all Trek-related activities, and transform into, say, Campus Jedis United. Now Star Trek fandom is no longer a collective interest of the members of Campus Trekkies United—because they did not ensure that the condition of a collective interest holds.

In fact, it would be more precise to say that Star Trek fandom was never a collective interest, only ever a shared one—because the condition of a collective interest never held in the first place!

The universal collective interest

A collective interest of Students Against a Democratic Society (ostensibly) is being against a democratic society. A collective interest of Campus Trekkies United (ostensibly) is being a fan of Star Trek.

But there is one sort of collective interest that will be present in any organization:

The continued existence of the organization itself.

Groups are how humans achieve their goals. Organization is power. It is in the interest of any member of an organization that the organization continue to exist. Any other shared interest may fail to be a collective one—except for this one.

Illusions

Suppose that a proper subset of a group’s members share a certain interest. This may be coincidence—nothing more than a consequence of base rates of that interest in the general population. But it may also be due to the fact that a proper subset of the group’s members itself constitutes a coherent group, which has collective interests of its own.

This also manifests in a more interesting way, as follows:

Suppose it is claimed that a certain interest is a collective interest of a given group. However, investigation reveals group members that do not share that interest.

The claimant(s) may cry “No true Scotsman”, “infiltrator”, etc. But another (and, it seems to me, more likely) explanation is that the claimed collective interest is indeed a collective interest—not of the whole group it’s claimed of, but rather of a proper subset of the greater group (which subset, however, may find it advantageous to be identified with the greater group).

(Finding examples of this dynamic is left as a fairly straightforward exercise to the reader.)

Previous post in series: Goodhart’s law.

“How do we split the loot?”

That was one of the biggest challenges of raiding in World of Warcraft.

We’ve gotten 40 people together; we’ve kept them focused on the task, for several hours straight; we’ve coordinated their efforts; we’ve figured out the optimal strategy and tactics for taking down the raid boss; we’ve executed flawlessly (or close enough, anyway). Now, the dragon (or demon, or sentient colossus of magically animated lava) lies dead at our feet, and we’re staring at the fabulous treasure that was his, and is now ours; and the question is: who gets it?

(:htoc:)

The problem of the indivisible

“Why not divide it 40 ways? That’s only fair!”

If only it were that easy! But the loot can’t be split 40 ways, because it’s not just a giant pile of gold coins; it’s (for instance): a magic warhammer, a magic staff, and a magic robe. Three¹ items; each quite valuable and desirable; each of which can be given to one person, and cannot be sold or traded thereafter. We have to decide, here and now, which three out of the 40 members of the raid will receive one of these rewards. The other 37 get nothing.

What to do?

It would be difficult to overstate how much thought went into answering this question, among WoW players; how much effort was spent on debating it; how much acrimony it spawned; and how critical was a good answer to it, in determining success in the most challenging endeavors in the game (high-end raiding). Disagreements in matters of loot distribution broke friendships, and ill-advised loot policies cracked guilds in half.

Nor should this be at all surprising, for World of Warcraft was human civilization in microcosm. The task of organizing and coordinating a raid was project management; figuring out how to take down a raid boss was strategic planning, and practical epistemology, and mathematics; making money at the Auction House (in order to purchase performance-enhancing potions) was economics. But the distribution of loot—that was politics.

Pie-slicing as project management

To the question of “how to split the loot”, there is at least one simple answer: a roll of the dice.² This method was used often—in “one-off” raids, wherein a group of players would come together, for this one occasion only, to defeat a challenge (or a connected group of challenges), and then disperse, entering into no longer-term relationship with one another, possibly never to cooperate or interact again. In such cases, it was understood that your participation comes with the promise of a chance at winning a reward. If the group happens to find some piece of treasure which you covet, you will have an opportunity to throw your name into the hat, and—should the Random Number God smile upon you—to win the prize. That’s all you can expect, and it’s no more than anyone else is getting. In a temporary collective, made up of strangers, one can hardly ask for more.

But such one-off groups are nowhere near effective enough to tackle the most difficult challenges that are available—that is, to do “progression raiding”³. For that, you need for the same 40 people to assemble, week after week, month after month; they must learn to work together smoothly, and they must all, together, learn the raid encounters, and the strategies and tactics for defeating them; and, just as importantly, all members of the raid must “get geared”—must acquire better and better equipment for their characters, in order to improve their performance and become better able to handle the next challenge, and the next. This is a sustained, collective effort, and it can only be managed by a persistent organization: the raid guild.

The challenges of running a raid guild are legion; there is much to say about them—enough for many more blog posts. For now, the key point is this: for a raid guild, the question of loot distribution is, at once, both a serious and thorny problem, and a powerful tool which may be applied to many other aspects of guild management.

There were many “loot systems”. Some were communistic: the raid leader (or a “Loot Council”, composed of the raid leader and a small handful of others) would simply decide which raid member would receive each piece of loot—“to each according to his need”. Others were at the opposite extreme—“free market” systems, where one accumulated “points” via raid attendance and contributions to the raid’s success, and the allocation of loot was decided via bidding.

Neither extreme ever sat well with me (for reasons that should be obvious enough to anyone with any passing familiarity with the real-world economic systems to which I alluded). When the guild to which I belonged decided to get serious about progression raiding, and it came time to formulate a policy for loot distribution, I advocated for a loot system which, to this day, I consider the most ideal, of all the systems I’ve encountered. The system was adopted, and it served us well for years to come. That system was known as the “Effort Points / Gear Points” system, or EP/GP.

Solution: EP/GP

The idea of EP/GP was simple. There are some actions/behaviors that you don’t want your members to engage in at all; those you ban outright, and set whatever punishments you see fit. Those aside, however, there are two categories of things that aren’t discouraged:

First, there are things that you want everyone to be doing as much as possible—things that are unboundedly good. Second, there are things which are good for people to be doing, healthy, expected, certainly not discouraged—but you don’t want anyone doing them too much, and you don’t want there to be a serious imbalance in who is doing those things.

The second category consists of things which people just want to do, of their own accord, and don’t really need to be incentivized to do; they’re their own incentives. The first category consists of things that you do generally need to incentivize people to do, even if people “want” (or want to want) to do them. (In WoW, the first category is “help the raid kill bosses” and the second category is “get gear, thus making your character more awesome”.)

The idea of the EP/GP distribution system is that you use the first category to rate-limit the second. Each member has two quantities associated with them: EP (effort points) and GP (gear points). Both start at 0; each goes up as a consequence of actions/behaviors in that category. Do unboundedly-good prosocial thing? Your EP goes up. Do self-incentivizing indirectly-prosocial self-benefiting thing? Your GP goes up. And whenever there is any scarce resource that people want, it is allocated according to EP/GP ratio; whoever has the highest such ratio gets the resource (and their GP goes up accordingly).

So, the more EP-generating things you do, the higher your priority in the allocation of rewards; the more GP-generating things you are allocated, the lower your priority subsequently. As long as you can define those categories, and place relevant behaviors/actions into them, EPGP works to allocate your scarce resources and incentivize members’ contributions.

The EP/GP system has a number of ancillary benefits:

First, new members immediately get the instant gratification of being top priority for resources, as soon as they contribute anything whatsoever (as EP and GP start at 0, any contribution makes EP positive, and positive/zero = infinite priority!) Having now given and gotten something of value, they are drawn in, at which point their priority goes down to below that of regulars/veterans; it fluctuates greatly at first, then stabilizes. This incentivizes early contribution, but doesn’t make people “pay dues” excessively to get anything at all.

Second, because EP can be assigned for anything, and relative values set to whatever the administration wishes, the system makes it easy to design incentive structures that encourage whatever you like

Third, there is tangible benefit to sustained contribution, without locking out newcomers.

A contrast: DKP

“Dragon Kill Points”, or “DKP”, was, once upon a time, the most popular loot system; it long predated EP/GP.⁴ In DKP, each member of a raid would receive some number of points (the titular “dragon kill points”) upon successful completion of a raid encounter. To receive a piece of loot, a raid member had to spend some of his points (the amount usually determined by a bidding contest among all raid members who wanted that item).

DKP had many faults, and waned greatly in popularity as WoW aged; better systems (such as EP/GP) had come along. With DKP, if you were a newcomer, joining a raid full of veterans, the only way you were going to get anything was if no one else wanted it—otherwise you had to toil through raid after raid, contributing effort but knowing in advance that you weren’t getting anything for your efforts (except scraps from the veterans’ table, as it were).

What makes a good loot system?

The twin needs, in any group that depends, for success, on a bunch of people all contributing as much effort as possible, are:

1. You have to pull in good people;
2. You have to get your people to stay, and keep contributing.

DKP was bad at #1, because the prospect of slaving away for weeks or months before you had accumulated enough to have a shot at the good stuff was daunting (and then you could be outbid by a veteran who’d been hoarding his points for longer; and even if you won, you might've just spent all your points on one thing; etc.). DKP was also bad at #2, because after you had accumulated a certain large pool of points, the incentive to keep contributing dropped off.

EP/GP, on the other hand, is good at #1, because your first reward is basically guaranteed, as soon as you contribute something of value. And EP/GP is good at #2, because going up in priority is easy at the start, and bouncing back from getting some gear is easy, and as it gets harder, well, ratios equalize; and as long as you keep contributing, you stay at a good ratio, and meanwhile, the higher your EP gets (if you’re a veteran), the faster it drops when you get something.⁵

Is EP/GP really the best way?

Later in WoW’s history—when it became possible for high-end raid encounters to be tackled by smaller raid groups—there arose, within some raid guilds, the practice of having multiple raid groups, including some that were more ‘elite’/exclusive than the guild’s main raid group. (Members of such smaller groups typically participated in the sub-group’s raids in addition to taking part in the guild’s primary raiding activities.) Where a good raid guild might’ve been in the 99th percentile of competence and achievement, among the overall player population, it might have within it a smaller raid group which was much, much further toward the right tail of the raid content achievement distribution.

Such smaller sub-groups usually did not use the EP/GP or other allocation system of the main group, but had their own, separate, loot policies. These policies typically skewed closer to “managed communism” than to “regulated capitalism” on the spectrum of loot systems; and I do not think that this is a coincidence. The members of these smaller, more exclusive groups—which, in virtue of their greater selectivity for competence and performance, almost always performed better and accomplished more difficult goals than a guild’s primary raiding group—exhibited a higher degree of sublimation of personal interest to group interest, than did members of a guild’s main raid group; they were more willing to make sacrifices “for the good of the raid”.

If you’re trying to maintain a raiding guild of 100 people, keep it functioning and healthy over the course of months or years, new content, people joining and leaving, schedules and life circumstances changing, different personalities and background, etc., then it's important to maintain member satisfaction; it’s important to ensure that people feel in control and rewarded and appreciated; that they don’t burn out or develop resentments; that no one feels slighted, and no one feels that anyone is favored. You also have to recruit new members, to keep up with inevitable member turn-over. All of these things are more important than “being maximally effective at taking down this raid boss right now, and then the next five bosses this week”. If you focus on the latter and ignore the former, your guild will break and explode, and people on WoW-related news websites will place stories about your public meltdowns in the Drama section, and laugh at you.

On the other hand, if you get 10 players together, and you say: “OK, dudes—we, these particular 10 people, are going to show up every single Sunday for several months, play for 6 hours straight each time, and we will push through absolutely the most challenging content in the game, which only a small handful [or sometimes: none at all] of people in the world have done”—that is a different scenario. There’s no room for “I’m not the tank but I want that piece of tank gear”, because if you do that, you will fail.

What a group of the latter sort promises—which a larger, more skill-diverse, less elite/exclusive, group cannot promise—is the incredible rush of pushing yourself—your concentration, your skill, your endurance, your coordination, your ingenuity—to the maximum, and succeeding at something really really hard, as a result. That is the intrinsic motivation which takes the place of the extrinsic motivation of “getting loot”. As a result, the extrinsic motivation is no longer a resource which it is vitally important to allocate. In that scenario, your needs are the group’s needs; the group’s successes are your successes; there is no separation between you and the group—and consequently, the need for equity in loot allocation falls away, and everything is allocated strictly by group-level optimization.

This was evident in the reactions people had, to seeing other people get loot. In a larger, somewhat-more-casual, raid group, it went like this:

Alice gets <awesome piece of gear>.

Alice: yay! :D

Bob: grats

[Bob is happy for Alice but also jealous, Bob wanted that thing too.]

In tighter-knit, more “hardcore” groups, it was more like this:

Alice gets <awesome piece of gear>.

Everyone in the raid: F*** YEAH!! :D

[Everyone is ecstatic that Alice got that thing; no one is jealous.]

In the latter case, it was not only understood, but viscerally felt, that every thing that anyone in the raid gets, is increased performance for the group as a whole—which is all that matters.⁶

World of Warcraft, especially WoW raiding¹, is very much a game of numbers and details.

At first, in the very early days of WoW, people didn’t necessarily appreciate this very well, nor did they have any good way to use that fact even if they did appreciate it. (And—this bit is a tangent, but an interesting one—a lot of superstitions arose about how game mechanics worked, which abilities had which effects, what caused bosses² to do this or that, etc.—all the usual human responses to complex phenomena where discerning causation is hard.)

And, more importantly and on-topic, there was no really good way to sift the good players from the bad; nor to improve one’s own performance.

This hampered progression. (“Progression” is a WoW term of art for “getting a boss down, getting better at doing so, and advancing to the next challenge; rinse, repeat”. Hence “progression raiding” meant “working on defeating the currently-not-yet-beaten challenges”.)

(:htoc:)

The combat log

One crucial feature of WoW is the combat log. This is a little window that appears at the bottom of your screen; into it, the game outputs lines that report everything that happens to or around your character. All damage done or taken, all hits taken or avoided, abilities used, etc., etc.—everything. This information is output in a specific format; and it can be parsed by the add-on system³.

Naturally, then, people soon began writing add-ons that did parse it—parse it, and organize it, and present various statistical and aggregative transformations of that data in an easy-to-view form—which, importantly, could be viewed live, as one played.

Thus arose the category of add-ons known as “damage meters”.

The damage meters

Of course the “damage meters” showed other things as well—but viewing damage output was the most popular and exciting use. (What more exciting set of data is there, but one that shows how much you’re hurting the monsters, with your fireballs and the strikes of your sword?) The better class of damage-meter add-ons not only recorded this data, but also synchronized and verified it, by communicating between instances of themselves running on the clients of all the people in the raid.

Which meant that now you could have a centralized display of just what exactly everyone in the raid was doing, and how, and how well.

This was a great boon to raid leaders and raid guilds everywhere! You have a raid of 40 people, one of the DPSers⁴ is incompetent, can’t DPS to save his life, or he’s AFK⁵ half the time, or he's just messing around—who can tell?

With damage meters—everyone can tell.

Now, you could sift the bad from the good, the conscientious from the moochers and slackers, and so on. And more: someone’s not performing well but seems to be trying, but failing? Well, now you look at his ability breakdown⁶, you compare it to that of the top DPSers, you see what the difference is and you say—no, Bob, don't use ability X in this situation, use ability Y, it does more damage.

The problem

All of this is fantastic. But… it immediately and predictably began to be subverted by Goodhart’s law.

To wit: if you are looking at the DPS meters but “maximize DPS” is not perfectly correlated with “kill the boss” (that being, of course, your goal)… then you have a problem.

This may be obvious enough; but it is also instructive to consider the specific ways that those things can come uncoupled. So, let me try and enumerate them.

The Thing is valuable, but it’s not the only valuable thing

There are other things that must be done, that are less glamorous, and may detract from doing the Thing, but each of which is a sine qua non of success. (In WoW, this might manifest as: the boss must be damaged, but also, adds must be kited—never mind what this means, know only that while a DPSer is doing that, he can’t be DPSing!)

And yet more insidious elaborations on that possibility:

We can’t afford to specialize

What if, yes, this other thing must be done, but the maximally competent raid member must both do that thing and also the main thing? He won’t DPS as well as he could, but he also can't just not DPS, because then you fail and die; you can’t say “ok, just do the other thing and forget DPSing”. In other words, what if the secondary task isn’t just something you can put someone full-time on?

Outside of WoW, you might encounter this in, e.g., a software development context: suppose you’re measuring commits, but also documentation must be written—but you don’t have (nor can you afford to hire) a dedicated docs writer! (Similar examples abound.)

Then other possibilities:

Tunnel vision kills

The Thing is valuable, but tunnel-visioning on The Thing means that you will forget to focus on certain other things, the result being that you are horribly doomed somehow—this is an individual failing, but given rise to by the incentives of the singular metric (i.e., DPS maximization).

(The WoW example is: you have to DPS as hard as possible, but you also have to move out the way when the boss does his “everyone in a 10 foot radius dies to horrible fire” ability.)

And yet more insidious versions of this one:

Tunnel vision kills… other people

Yes, if this tunnel-vision dooms you, personally, in a predictable and unavoidable fashion, then it is easy enough to say “do this other thing or else you will predictably also suffer on the singular metric” (the dead throw no fireballs).

But the real problem comes in when neglecting such a secondary duty creates externalities; or when the destructive effect of the neglect can be pushed off on someone else.

(In WoW: “I won’t run out of the fire and the healers can just heal me and I won’t die and I’ll do more DPS than those who don’t run out"; in another context, perhaps “I will neglect to comment my code, or to test it, or to do other maintenance tasks; these may be done for me by others, and meanwhile I will maximize my singular metric [commits]”.)

It’s almost always the case that you have the comparative advantage in doing the secondary thing that avoids the doom; if others have to pick up your slack there, it’ll be way less efficient, overall.

Optimization has a price

The Thing is valuable, yes; and it may be that there are ways to in fact increase your level of the Thing, really do increase it, but at a non-obvious cost that is borne by others. Yes, you are improving your effectiveness, but the price is that others, doing other things, now have to work harder, or waste effort on the consequences, etc.

(Many examples of this in WoW, such as “start DPSing before you’re supposed to, and risk the boss getting away from the tank and killing the raid”. In a general context, this is “taking risks, the consequences of which are dire, and the mitigation of which is a cost borne by others, not you”.)

Then this one is particularly subtle and may be hard to spot:

Everyone wants the chance to show off their skill

The Thing is valuable, and doing it well brings judgment of competence, and therefore status. There are roles within the project’s task allocation that naturally give greater opportunities to maximize your performance of the Thing, and therefore people seek out those roles preferentially—even when an optimal allocation of roles, by relative skill or appropriateness to task, would lead them to be placed in roles that do not let them do the most of the Thing.

(In WoW: if the most skilled hunter is needed to kite the add, but there are no “who kited the add best” meters, only damage meters… well, then maybe that most skilled hunter, when called upon to kite the add, says “Bob over there can kite the add better”—and as a result, because Bob actually is worse at that, the raid fails. In other contexts… well, many examples, of course; glory-seeking in project participation, etc.)

Of course there is also:

A good excuse for incompetence

This is the converse of the first scenario: if the Thing is valuable but you are bad at it, you might deliberately seek out roles in which there is an excuse for not performing it well (because the role’s primary purpose is something else)—despite the fact that, actually, the ideal person in your role also does the Thing (even if not as much as in a Thing-centered role).

Five quick tips for perfect cheesecakes:

Room temperature ingredients

Cream cheese, sour cream, heavy cream, eggs, and so on—all of these should be allowed to come up to room temperature, before mixing them to make the cheesecake batter.

Don’t overbeat

Once you start adding the eggs, don’t overbeat the batter. Use the lowest speed on a hand mixer; run it until just combined; stir/fold the rest of the way by hand, with a spatula. Watch out for bubbles—you don’t want air incorporated into the batter.

Grease the springform

Butter the sides of the springform pan, or line the sides with parchment paper. This is so that the cheesecake doesn’t stick to the springform pan frame; if it does, then when it shrinks in the fridge, it can crack. Also, it will be much harder to remove the frame if the cheesecake has stuck to it, and you may damage the cake in the process.

Water “bath”

Cheesecake recipes tell you to use a water bath, or bain marie, to moderate the temperature of the cake as it bakes, and maintain a humid environment—these things prevent cracks. This is good advice.

But what if your springform pan leaks? Do you have to wrap the bottom in aluminum foil? Sometimes water can leak through anyway…

Forget those worries! Instead of actually placing the springform pan in the pan of water, put the pan of water on the oven rack below your cheesecake! This works just as well, and guarantees zero leaks.

Leave the cheesecake in the oven

When you turn off the oven, don’t take the cheesecake out immediately! The shock of the rapid temperature and humidity change may cause cracks. Instead, leave the cheesecake in there. Let it sit there for another 2 hours; only then take it out, let it cool on a rack, and then put it in the fridge to chill overnight.

BONUS TIP

Suppose all these tips fail you, and your cheesecake still cracks, or looks imperfect. What to do?

Traps often seem like they exist to do the following:

1. Force the players to say "we check for traps" (or convince/threaten their DM into accepting "we are constantly and always checking for traps!" as a valid "standard operating procedure").
2. Slow down the party's progress, due to constantly and always checking for traps. (This allows the DM to make more wandering monster checks, thus affording more opportunities to cackle with sadistic glee when the party is jumped by a … <rolls d20> … "no encounter".)
3. Punish players who don't do #1, by inflicting gruesome injury on their characters, and forcing the party cleric to spend valuable spell slots to heal them.
4. Force someone in the party (inevitably, the hapless rogue) to spend ranks on Disable Device (the other use of the skill—opening locks—is fully subsumed by the knock spell, rendering rogues useless, although of course they were already useless, right?).
5. Reduce the party's successful negotiation of the fearsome dungeon hazards known as traps—those intricate brainchildren of the dungeon creator's fiendish intellect—to simple, unadorned die rolls (Disable Device again). How good are you at rolling high numbers on a d20? Pretty good? Great, you can advance through the dungeon. Not so good? Sorry, you have to keep rolling until you roll a high number. But if you roll low enough, something terrible may happen to your character!

Some of the above is true, some of the time. Sometimes it's not. But it usually feels true. Why?

Well, how does interaction with traps usually go?

DM: You are in a hallway. What do you do?
Players: We advance! <seeing the DM hefting a d20> … CONSTANTLY checking for traps, of course!
DM, rolling: You find a trap!
Players, with feigned excitement: Great! Good job, rogue!
DM: <cursory description of the trap> [this part is optional; sometimes it's omitted]
DM: Roll to disable the trap.
Rogue: <rolls>
DM: You have disabled the trap!
Players, apathetically: Yay.

(What was that phrase? Something about a Mechanics' Association…?)

Other variants exist. Sometimes a DM rebels against the above blandness, and tries to "make traps more interesting" by describing their mechanisms and workings in detail. Having done this, the DM looks at the players expectantly, only to find himself having this conversation:

DM: ... and that's what the trap is like. What do you do?
Rogue: I disable it!
DM: How?
Rogue: … with my Disable Device skill, how else?
DM: Ok, but, like, specifically, what do you do? Describe it.
Rogue, exasperated: I don't know, man! I'm not my character! Figuring out how to disable it is my character's job, not mine. What, do you make the fighter describe how exactly he swings his sword when he makes an attack roll? I invested fifteen ranks into Disable Device, my total modifier with all my bonuses is +27, so… I disable it. Somehow. My character's really good at this, he figures it out.
DM: … sigh.

Some DMs give up, at this point. Others, dead set on the notion that traps must be "interesting" but unable to counter the assertion that "my character's really good at this, he figures it out", describe the trap in detail—then, upon a successful Disable Device roll, also describe how the rogue disables the trap. This sort of "dialogue" is usually met with understandable blank stares from the players (watching a DM play D&D against himself is, at best, a slightly bizarre experience), while making the DM question his life choices.

But wait, more absurdity lies ahead! Even if the DM decides to continue performing the solo trap-description act, it all falls down when magic traps arrive on the scene.

The rogue, entering the hallway at the head of the party, spies the symbol immediately, and says, "Hold, friends! A magic trap lies ahead! Wait here, while I disable it." His player then rolls a Disable Device check… beats the Difficulty Class of the trap… and the rogue…

Does what? What does it mean to disable this trap? How is it done? Is the rogue disabling it at range, from more than 60 feet away? How? Is that even a thing the Disable Device skill lets a character do? What if the trigger radius was 600 feet? A mile? Does the rogue approach the trap? But that'll make it trigger. Won't it? Does the rogue somehow render himself undetectable to the symbol's magic? How? Is that an ability that rogues get? (Is that what "trapfinding" is?) Does it allow the rogue to simply walk past a trap without disabling it? Is this power applicable in other, non-trap-related, situations? If not, why not? Is there a special dance the rogue does, as he approaches the symbol, that makes it not activate? What is going on here?

Trap disabling is a dissociated mechanic. Dice are rolled, game mechanics are used, but there is no sensible way to translate any of that into a consistent, coherent description of what actually is happening in the game world. Oh, certainly in any particular case a description may be concocted, but it is a post-hoc thing, which may sound plausible at first blush — but either has no bearing whatever on what actually happens in game terms, or it ends up only complicating things (after all, what happens when this explanation turns out to make no sense for the next magic trap?). (Justin Alexander's commentary on dissociated mechanics applies in full here, and I can express his points no more eloquently than he.) To avoid having to face this nonsense head-on, most DMs simply handwave away the details, saying "you disable the trap", and hurry on to the next part of the action. And because doing this is inescapable for magic traps, a DM will usually end up doing the same thing for all traps, having finally given up on "interestingness".

This has a corrosive effect on the game. It breaks immersion: we watch as our characters move through their world, the camera view of our mind's eye following them closely, panning over the critical details of their surroundings, showing us their actions in full Technicolor — until it comes time to handle a trap, and the director says to us: "uh, now, we're going to look away, and blur the view, and, uh, something happens, don't think too hard about it, ah, there we go, all done!" This damages the players' ability to imagine themselves in their characters' place—to roleplay—and reminds them that this is all fake. And, of course, it leads directly and inevitably to that laundry list of complaints at the beginning of this post. If traps have no reality or meaning within the game world, it's hard to see them as anything but annoyances. (In fact, the more the players are pulled out of immersion within their characters and within the world the characters occupy, the more likely it is they will start to see all obstacles facing their characters as annoyances to no good purpose.)

Ok, so what's the alternative?

Obvious answer: don't use traps. At all. This does work, in that it avoids all of the problems just described. Naturally, this is not the answer I chose.

…

So, again: why traps?

Tradition: Traps have long been so much a part of D&D that their existence and effects are encoded in the design of classes, of spells, of entire campaign worlds. What's a dragon's lair without traps? A lich's stronghold on the Outer Plane of Acheron? The lost tomb of ancient kings? What's White Plume Mountain without the razor-blade pits in the frictionless room, or newbie sorcerers drowning face-down in the mud thanks to a symbol of sleep and a hungry sphinx? What's Castle Ravenloft without a party member being silently teleported into a crypt full of wights, and a disguised wight installed in the victim's place?

Texture: Traps lend a sense of peril to a dungeon delve, the feeling of being in a dangerous place, where you are an unwelcome intruder, and the place itself wishes you ill, and will inflict harm upon you, if you let it. A trap-filled dungeon is not merely another place to have some combats. Pursue the dragon to his lair, and you'll find the tables turned against you; the dragon's mastery of his domain makes your every step potentially your last. The lich whose phylactery you seek has spent a century making his home into death for unwanted guests, and you should fear entering within, even after you've destroyed its owner. Within the peak of Elbrus lies a stronghold of the spell weavers, a strange place of alien geometry and bizarre devices which may do only the gods know what; every object, every surface, is a risk to touch. At its best, a dungeon, creatively furnished with devious traps, can take on a life of its own; it can have a personality, remembered long after its inhabitants have been defeated and forgotten.

Immersion: In a world with teleporting demons and fireball-slinging wizards, the player characters' actions may start to seem increasingly unreal. The characters wander through exotic realms, engage in larger-than-life battles with fantastic monsters, use magic to counter magic, and ward themselves with more magic. The world around them seems to blur and become indistinct, not quite real unless initiative is rolled, painted only in broad strokes otherwise. Traps, done well, force the players to engage directly with the game world, which appears now in sharp focus. Other things may have this effect also—perilous environmental conditions, realistic weather, consistently vivid sensory descriptions—but traps are visceral; they hit home.

Traps tend to have very physical effects: they drop a character down a shaft onto spikes, they send her careening through a slippery chute, they stab at her fingers with cutting blades, they drop stone blocks on her head. The experience of a trap's effects is much easier for the player to imagine from the character's viewpoint than is being affected by a harm spell. This pulls the player into the game world, and into the character's head. For the same reason, well-designed and well-described traps are interactive and interactable: a player can imagine things her character can do with, to, or in a trap, because the trap itself is something imaginable; and the very physicality, the three-dimensionality, of this piece of the game world, means that there is a wide variety of things for a character to do with it.

Challenge, choice, control: People don't just play D&D to be challenged, but challenge is a big part of the game. What is gotten too easily, what is given for nothing, unearned, is not valued; for lack of challenge, all "rewards" taste bland; false "accomplishments" are empty words. And overcoming challenge is its own reward, for it means validation of one's talent and skill: success in D&D demonstrates cleverness, creativity, flair, and other virtues besides. Diversity of challenge both prevents satiety and boredom, and ensures that everyone gets what they like best, at least some of the time. Traps add a new dimension of challenge, one that is unlike the others in the game. And diversity of challenge is itself challenge, forcing characters and parties to be more the Renaissance Man than the narrow specialist.

But "challenge" is not merely "difficulty" or "complexity" or "obstacle". A challenge must be comprehensible, and it must be solvable. To be satisfying, it must allow real, interesting choices; interesting, because their potential outcomes are knowable, and real, because those outcomes are truly different from each other. And a challenge is most exciting to encounter, and most satisfying to solve, when the solution is one of which, and during which, the solver is entirely in control. Such challenges become sought-after opportunities to exercise control, to use one's skills at their best; to win. Well-designed traps, encountered within a game system well-designed to handle them, have all of these qualities.

…

So what kind of game system is that—"well-designed to handle traps"? The old-school, perhaps? How were traps handled in the olden days? Well, first the trap had to be found; without the Search skill, that meant describing your character's actions: "I walk forward slowly, tapping my staff on the floor before me, side-to-side across the width of the corridor"; "I carefully rap each section of the wooden door with my gauntlet, to see if any part sounds like it might be hollow"; etc.¹ The DM might say, "One of the floor tiles just ahead wobbles a tiny bit as you tap it." Further investigation suggests it's a pressure plate, a likely trigger for a trap, of what sort—the PCs don't know. The players brainstorm a solution. "We'll jam it! Wedge a few wooden stakes at the edge, that'll fix it so the plate won't lower!" They announce that their characters do this very thing—"yep," says the DM, "you've wedged it real good"—and then walk forward. Then it's up to the DM: did it work? Or is hilarity about to ensue? The players will soon find out, but anyway it's out of their hands… Meanwhile, the DM is laughing inside, at all the ways in which the plan is foolish, and impractical, and anyway doomed from the get-go.

There are three problems with a system like that, one of which is not a problem, and two of which are. Problem one: in the old school, a 20th level rogue is just as good at finding and disabling traps¹ as a 1st level rogue, or a 1st level fighter; because this sort of challenge doesn't rely on the character's numbers, it doesn't scale with character level.² This is not actually a problem. Where is it written that all challenges in the game must be directly dependent on character level? If a lower-level character can defeat a trap, with clever play instead of abstract numbers, as well as a higher-level one can, then call that a feature of the system.

Problem two: action resolution is entirely at the whim of the DM; he determines, by fiat, whether a trap will be found, and whether the players' method of disabling it will work. The DM might justify his ruling to the players, but it's still fiat. This may be fine for some things, but for interactions with complex, physical systems, like traps, it's strange, and often makes the players' mental image of the game world diverge from what's in the DM's head. Worse, while the DM may well be a walking perfect physics simulator, the players must also trust in this fact; if they doubt it, then it begins to seem as if the challenge is not "solve this trap", but "be persuasive to the DM". That's not a fun game for anyone.

Problem three: some players just plain don't like the cognitive challenge of visualizing a trap, based on the DM's description, analyzing it, brainstorming ways to disable it, and determining the best solution. Fine and well; people enjoy different things. Does this mean that traps, in their entirety, that whole part of the game and all its implications, can never be fun for such players?

…

I say that these problems can be solved. Device lore (see below) is part of my solution.

The full solution looks like this: The Disable Device skill represents the physical skill of jamming levers and wheels, threading wires through mechanisms, striking a clockwork device just right, so the gear shafts warp and the device misfires. It's training, practice, muscle memory; in other words: execution. Disable Device does not reveal, nor substitute for, a plan for just what the character is trying to do: disable the trap? How, exactly? Say what your character does, and roll Disable Device to find out how well he does it; but the skill doesn't give you the "what". (After all, Disable Device is tied to Dexterity, not to Intelligence!)

That's where device lore comes in. A successful device lore check tells you what you can do to "solve" the trap, what options you have. Higher rolls reveal more options; simpler solutions, more convenient ones; solutions that don't involve modifying or breaking the device at all. Device lore steps in for the player's need to analyze the trap himself. The player rolls the check, the DM reports the results, and the player considers the options thus revealed and decides what his character will do; then, Disable Device determines how well the characters does it.

Thus when a party comes across a trap, their interaction with it can go in one of four ways:

Scenario 0: The characters don't see the trap before blundering into it. The trap is triggered, and the PCs are exposed to its effects. Perhaps some party members make their saves, if the traps allows a save; perhaps not. In any case, the consequences of the trap's triggering take place.

In the other three scenarios, the characters detect the trap (by using the Search skill or otherwise); the DM then thoroughly describes just what the characters find, what they see—the physical nature of the trap. Then–

Scenario 1: The players analyze the trap, based on what their characters can see of it, and come up with a solution: a way to disable it, or "solve" it in some other way. The characters carry out the players' plan. How well they succeed now depends on the dice, the number of Disable Device ranks of the party's trap-handler, the complexity of the trap, and other factors of a game-mechanical nature.

Scenario 2: The party's rogue³ rolls a successful device lore check. The DM provides the rogue's player with their character's analysis of the trap, which includes one or more potential solutions—approaches to disabling or bypassing the trap. If the check result was high enough to get multiple potential solutions, then these may differ in success chance, in time required to implement them, in resources which must be expended, in danger, and in any number of other things. This all the DM also conveys. The players, weighing their options, decide which approach to take. The characters carry out the players' chosen plan. How well they succeed now, again, depends on the dice, and the rogue's (or another party member's!) Disable Device ranks, the complexity of the trap, and all the same other factors as in the first scenario.

Scenario 3: The party has no rogue³, nor are the players able (or willing) to analyze the trap and solve it themselves. The party cannot disable the trap, nor can they rig the trap to bypass it safely. They must find some other way of handling this challenge. Perhaps the wizard casts dimension door to teleport the party past the trapped hallway; perhaps the bard uses a staff of passage to create a phase door through the wall; perhaps the rogue pulls a bear out of her bag of tricks and has it trigger the trap, or the fighter triggers it himself—how bad can it be? Or maybe the characters simply decide to avoid this part of the dungeon, or this treasure chamber, for the time being.

The last piece of my solution is more subtle, made of the confluence of many factors. In the original, Gygaxian, vision⁴, traps are not a "hard barrier", which the PCs must disable if they're to progress through the dungeon, or even survive it. Traps happen; the clever and cautious party avoids most of them, but even for the most careful dungeon-delvers, some traps are unavoidable. The characters must simply deal competently with the trap's effects, and any one trap is not a game-ender; real failure only comes when the party takes a reckless attitude toward the whole dungeon, falling for trap after trap, and eventually succumbing.⁵ And in the Worlds of Adventure campaign, not even death ends the game; after losing one of their number to a careless encounter with a scything blade, the party returns to the City and has the character resurrected—an expense covered by resurrection insurance from the Argos Trade Consortium, which the prudent adventurer has, of course, seen fit to purchase. And if not… well, there's always a new character to roll. Isn't there a monk you've been wanting to try?

To summarize: traps in the Worlds of Adventure campaign are a common form of challenge, to be handled by the players and the characters, with robust game mechanics designed for the purpose. Traps, just like other challenges in the game, test the players' creativity and cleverness, and their skill at building effective characters and effective parties. The characters' game-mechanical abilities work in concert with the players' ingenuity; greater weight may be put on the former or the latter, depending on how the players choose to play, and how they've built their characters. Traps, as integral, physical parts of the simulated game world, may be be handled in many ways, on many levels—with trap-specific game mechanics, with general character abilities (magical or mundane), with strategic and tactical thinking—according to the players' preferred style and approach. The player characters may thus expect to regularly encounter traps in their excursions into dungeons across the worlds.

Postscript: Magical Traps

Many parts of the solution outlined above speak of physical or mechanical traps, but the device lore ability mentions magical traps as well. What of them?

Magical traps are of two kinds: spells that create trap-like effects, and non-spell traps that have magic triggering "mechanisms", magic effects, or both. The first kind is generally well enough described (as are most spells in modern D&D-type systems) that they may be treated much like mechanisms: the spell has a clearly described effect; how spells interact with other things is also well defined; thus the players can easily reason about the spell and the trap it creates.

The second kind—"traps" where the trigger, the effect, or both, are magical in nature, and do not necessarily duplicate the effects of any spell (this is most relevant in the case of the trigger)—should, in some cases, not be properly thought of as traps (in the narrow game-mechanical sense) at all. Rather, they are perhaps magical phenomena, that happen to have detrimental effects. They are probably not "disabled" or otherwise handled in anything like the same way that the usual sort of traps are. Perhaps they can be dispelled or disjoined, perhaps a magical keystone must be found to turn them off, or something else… in any case, it's a different category altogether.

In other cases, where the trap is magical in construction, but quite trap-like in behavior (e.g. magical runes that cover every surface of the treasure chamber, that fill the room with sheets of lightning if any words are spoken within), it may still not be possible to disable it; magical means of circumvention (dispel magic, etc.) may be required. In any case, a successful device lore check will reveal these facts about the trap (as may a Spellcraft check).

Appendix: Device lore

A rogue may make a special device lore check, with a bonus equal to his rogue level + his Intelligence modifier, to see whether he can comprehend the nature and workings of a trap, and come up with ideas on how to disable it, or understand how to bypass it. (Device lore also applies to mechanisms and devices that are not traps.) (If the rogue has 5 or more ranks in Knowledge (engineering), he gains a +2 bonus on device lore checks concerning mechanical traps, but not magical ones.)

A successful device lore check will not allow the rogue to disable or bypass the trap, only reveal methods for doing so. A successful Disable Device check is still needed in order to disable a device. (Bypassing a trap or other device may involve the use of other skills or abilities, or no skill checks at all, depending on the specific nature of the device and other aspects of the situation.)

The Difficulty Class for the device lore check depends on the type and complexity of the device or trap, as shown on the table below.

Magic Traps: Most magic traps cannot be disabled via mechanical means, only bypassed. It may be possible to dispel or otherwise counter them with magic, however. A successful device lore check reveals all of this information about a given magic trap. Additionally, making a successful Spellcraft check against the device lore DC of the magic trap reveals the magical properties of the trap (see the Spellcraft skill description for details), and grants a +2 bonus on the device lore check.

For every 5 points by which the device lore check result beats the Difficulty Class of the device, it reveals another method for disabling or bypassing the trap. Higher check results reveal superior (quicker, more efficient, etc.) methods.

It takes one round to make a device lore check. A rogue may take 10 or take 20 on the check; the normal rules for taking 10 and taking 20 apply.

A character without the device lore ability may make an "untrained" version of this check (which is just an Intelligence check), but only for devices of moderate or lower complexity (device lore check DC of 15 or less).

A character with the device lore ability also gains a +2 competence bonus on Search checks to discover relatively small mechanical traps and devices such as poisoned needles, spring blades, and the like. This bonus increases to +4 at 10th level.

(Yes, there is a recipe for Key lime pie, down near the end of the post. You can skip to the recipe if that’s all you’re here for!)

(:htoc:)

“What’s so great about your new X?”

A recent comment I wrote on another forum described how to convince people of the value of a technique. Read the comment for details, but the short version is: display something which you made by using the technique. If the thing is clearly cool, then the technique recommends itself—“by their fruits ye shall know them”.

But what if the technique is a meta-technique? That is, what if the things you make with it are themselves techniques? A technique is not obviously cool, that being our original problem…

Again the solution is simple: go down another level. Display some cool thing you have made or done, using a technique which you have made by using the meta-technique. Thus the first-order technique recommends itself by its product; and the second-order technique recommends itself by its product, which is the first-order technique.

(Thus you should never have to say anything like “but the things that this technique makes are intangible, so how can I show them to you?”.)

That’s the claim, anyway; but maybe you think I ought to put my money where my mouth is? Well, indeed; and thus, this post.

Now I’m going to switch gears and talk about pie.

This section starts with a picture of a pie

The garnish is lime zest.
(click to enlarge)

That’s the pie. I made it, today. It’s delicious, and was quite easy to make.

Some more pictures:

To make a Key lime pie, you need a recipe for Key lime pie. But there are many recipes; how do you choose among them? (Do you choose? Is inventing your own recipe an option? But how do you do that?)

Later on in this post, I’m going to talk about how I made the pie (i.e., I will provide the recipe). First, though, I’ll tell you how I formulated the recipe (I didn’t just use a recipe I found somewhere!). But before that, I’ll tell you how I came up with the technique which I used to formulate the recipe.

You read that right: I propose to take you on a journey up and down fully three levels of meta. (And all for pie! Is it worth it? Yes. Yes it is.)

SDI: Not just for shooting down Soviet missiles

Pictured: not a Key lime pie.

Alicorn of Less Wrong described the “Saturation, Distillation, Improvisation” approach to developing procedural knowledge. (She, too, used it to come up with a cookie recipe. Coincidence? Or proof that dessert-baking is the most rational activity?)

SDI has three steps. Quoting/paraphrasing the key parts:

1. “[C]ollect procedural instructions for the object of your search from many different people, saturating your field of search with a variety of recommendations.”
2. See what the instructions have in common, and also determine whether to include any non-common elements.
3. Try out the resulting procedure, then experiment with variations on it.

So that’s the meta-meta-technique. I’m going to take the SDI framework, and refine it into a more concrete, more systematic form. This meta-technique is one which I’ve applied before (I used it to develop a killer brownie recipe, for example, and use an abbreviated form of the technique often when experimenting with recipes); today we’re going to apply it to Key lime pie.

The pie matrix (saturation)

I pulled a bunch of Key lime pie recipes off the internet. No special method was used to collect these; I just clicked on most of the first page of search results (making sure to include reputable sites like Serious Eats, and widely-used ones like Allrecipes).

Because Key lime pie consists of three components—the crust, the filling, and (optionally) the topping—which are mostly independent in terms of preparation (and thus, which it should be possible to optimize separately), I also included the graham cracker crust recipe from Christopher Kimball’s Dessert Bible (which I have used, to good effect, in many other recipes).

I put all the recipes into a table (a spreadsheet ), for easy comparison.

(All measurements have been normalized.)

Compare & contrast (distillation, part I)

I’ll do the ‘distillation’ step in several parts: I’ll examine, separately, what the recipes say about the ingredients for the crust, the filling, and the topping. If the recipes differ in any other ways (such as in the parts of the preparation process that aren’t separable, e.g. in how to assemble the components), I’ll note that as well.

Ingredients

Crust

Graham cracker crust before baking.
(click to enlarge)

Two of the recipes—both the ones that come from Allrecipes.com—lack ingredients or instructions for preparing a crust, because both recipes instruct you to use a pre-made, store-bought crust.

We’ll be ignoring those, of course. A store-bought crust?! Absurd. Is it rational to use an inferior, mass-produced crust, instead of making one from scratch with your own two hands? I submit that it is not rational.

That aside, all the crust recipes are fairly similar:

• They all use graham crackers (as opposed to some other sort of cookie or cracker). (This is what a single graham cracker looks like.)
• They all use almost identical amounts of butter (ranging from 4 tbsp to just over 5 tbsp).
• The amount of sugar varies from 3 tbsp to just over 5 tbsp (though note that the Dessert Bible recipe calls for brown sugar instead of white).
• Only one recipe explicitly calls for adding salt to the crust… but that may not mean much (see footnote 2).
• The Pioneer Woman recipe calls for a whopping 18 graham crackers; the others vary in a smaller range (from 9 to 11).
• The recipes are all prepared in similarly sized pans (all but one recipe are made in a 9″ pie dish; and an 8″ springform pan will not be significantly different in volume).
• All the crusts are prebaked at 350°F, except for the Serious Eats crust, which is prebaked at 375°F.

Filling

Filling, prior to baking.
(click to enlarge)

The filling, made with lime juice and sweetened condensed milk, is the star of the dessert⁹; and here we begin to see some interesting variation between recipes.

Comparing the filling recipes, we may notice at once that all but one calls for 14 ounces of sweetened condensed milk. That’s an awfully specific amount to be so consistent across so many recipes. What’s going on?

Why are they all the same size…?
(click images to enlarge)

One of the recipes—recipe (b) from Allrecipes—calls for a dramatically different amount of sweetened condensed milk (24 oz.); and we can see at once that this recipe is, in other ways also, the odd one out. It calls for no egg yolks at all, but is unique in calling for sour cream (a half-cup of it).

(An aside: consider what would happen if you decided to make a Key lime pie for the first time, searched for recipes, and chanced to choose this one, without doing any systematic comparison of recipes. Clearly, this recipe is quite different from most Key lime pie recipes; and the resulting pie is sure to be different as well. Having made the pie according to Allrecipes.com recipe (b), what would you have learned about how to make Key lime pie? Well… not much. It is so much of an outlier that having followed it, and having seen what comes of following it, simply doesn’t tell you very much about what any of the more ‘mainstream’ Key lime pie recipes are like. This is one of the reasons the SDI method is important: to ensure that you get the “lay of the land”—that you have a coherent and reasonably complete grasp of the state of available procedural knowledge of the thing you’re attempting, and what variations exist in this landscape.)

Leaving the outlier aside, the filling recipes are reasonably similar to each other:

• Amount of lime juice varies from 4 oz. (the most common amount) to 6 oz.
• Amount of lime zest is almost always 1 tbsp, although one recipe uses only 2 tsp (remember that 3 tsp = 1 tbsp), and one recipe omits it entirely.
• The egg yolks are the most widely variable ingredient; their number varies from 2 to 5 (though differences in egg size—and the fact that many recipes do not specify what size eggs to use—make it difficult to do a totally direct comparison)

Topping

Whipped cream topping.
(click to enlarge)

Key lime pie is generally topped with whipped cream, and then (optionally) garnished with a sprinkling of lime zest. Of course, the topping may be skipped; a Key lime pie is technically still a Key lime pie even without whipped cream on top.

But what kind of operation am I running here—some penny-pinching diner? No! Here at Cognitive Pie, we go for broke. Of course there’ll be whipped cream, and it will absolutely be garnished with lime zest.

Whipped cream topping (a.k.a. “whipped cream, plain and simple”) is so simple to make that several of the recipes either do not provide ingredients or instructions for it at all (only mentioning that it may be added, if desired), or provide only vague directions. Among those recipes that do tell us how to make whipped cream, however, there are several notable things.

First, only one topping recipe (from Simply Recipes) calls for vanilla extract (though The Pioneer Woman mentions that vanilla may be added at your option)—and a small amount, at that: 1⁄8 tsp.

Second, all of the recipes call for confectioner’s (a.k.a. powdered) sugar, not granulated or caster sugar.

Finally, one recipe—the one from Serious Eats—adds an interesting optional ingredient: 1⁄4 cup mascarpone (a soft, spreadable Italian dairy product that’s somewhat similar to cream cheese, but is less sour, and has a more complex flavor).¹⁰

Process

Crust

Making a graham cracker crust is simple, but we can see one major variation among the recipes: whether or not the crust should be chilled in the refrigerator prior to baking. Two of the recipes say yes; the others skip this step.

Other than that, all the recipes instruct us to make the crust in the same way:

1. pulverize crackers (using a food processor is usually, and quite reasonably, suggested for this)
2. add sugar
3. add melted butter
4. press into pan
5. chill (in the case of two of the recipes)
6. prebake (then remove from oven, to add the filling to it).

Nearly all the recipes tell you to bake the crust at 350°F, except for Serious Eats, which specifies 375°F. Baking times, however, vary substantially—from 5 minutes all the way up to 15 minutes (in the case of the Dessert Bible standalone recipe for graham cracker crust).

Filling assembly

The filling for Key lime pie is also quite straightforward, having less than a handful of ingredients and requiring no complex processing. Most of the recipes simply instruct you to “mix” or “combine” the ingredients (one recipe—the Serious Eats one—tells you to do this with the food processor; the others say to mix in a mixing bowl). Two of the recipes, however, tell you to beat the egg yolks by themselves, until light and fluffy, and only then to add the other filling ingredients. In either case, the filling is then poured into the crust, which is then returned to the oven for baking.

Baking

Baking temperatures for the filling vary from 325°F to 375°F. Baking times vary from “5 to 8 minutes” on the low end to “14 to 17 minutes” on the high end.

Topping

Several of the recipes give fairly detailed instructions for making the whipped cream topping, while others leave it as an exercise for the reader. I won’t enumerate the differences here, because, for the most part, whipped cream is whipped cream, and one can find instructions for making whipped cream anywhere; they will not be particular to Key lime pie recipes (since the pie, at this point, is already baked, and the topping is simply placed on top of it).

Post-processing

All but one recipe instructs you to chill the pie in the refrigerator before serving (for varying amounts of time, from “1 hour” to “overnight”; though most recipes suggest that longer is better). One recipe, however, also tells you to freeze the pie for 15–20 minutes before serving.

Analysis (distillation, part II)

Here I depart from the SDI method as described by Alicorn, by adding an extra step to the ‘distillation’ of the collected knowledge. In this part—analysis—I closely examine what I’ve learned via ‘distillation’, and I make sure that I understand, as specifically and as comprehensively as possible, why each recipe is the way it is. What is each ingredient doing in the recipe? Why that much, and not more, or less? If an ingredient is present in one recipe that is absent in another, why is it there? And if vice-versa, why is it missing? What is the cause of the variations in quantities, both small and large? How much is explained by chemistry, how much by convenience, how much by preference, and how much by happenstance?

It is not always possible to know, with certainty, the answers to all of these questions. But the more of them I can answer, the more solid will be the ground for the next step: synthesis (a.k.a. ‘improvisation’). Once I know why things are they way that they are—and why they are sometimes one way, and sometimes otherwise—then I will know enough to judge how they should be. These judgments will be based on my predictions of what will yield the best result (in cases where the result may be evaluated more or less objectively), and also on my own preferences (which may—indeed, almost certainly will—differ from the preferences of many other bakers and recipe-writers).

Aside: background knowledge

We have already seen how background knowledge—of ingredients, of baking techniques, etc.—is helpful, or even necessary, when ‘distilling’ the collected knowledge. If you did not know that graham crackers come in boxes of three packets, or that sweetened condensed milk is sold in one particular size of can, or that most pies are made in 9″ pans, then you would have, by now, been led astray several times. (Confusion about ingredient amounts, in particular, can easily ruin a recipe.)

In the ‘analysis’ part of the process (and even more so in the ‘synthesis’ part), background knowledge becomes even more critical, as we’ll see. It’s simply infeasible to apply this sort of technique if you start from a place of zero knowledge of baking. (I expect that this generalizes to other domains as well.) The remedy for this, in my experience, is to bake many desserts, trying out many recipes; to read detailed explanations such as this blog post (or, even better, cookbooks such as the Dessert Bible, which contains both detailed background information and a slew of easy-to-follow and well-explained recipes); and to discuss the subject with other bakers, of various levels of experience.

Crust ingredients

(click to enlarge)

There is not much to say about the crust of a Key lime pie that can’t be said of any graham cracker crust. Certainly it’s delicious. The butter softens the cracker crumbs and makes them stick together; the sugar sweetens it. The crust hardens as it bakes, turning into something like one giant, pie-shell-shaped cookie.

Chilling the crust

Why chill the crust before baking? It’s not clear. In the two recipes that include a chilling step, the crust clearly requires more baking (a higher temperature in one case, a longer baking time in another). Isn’t it a wash, then? Couldn’t you not chill the crust, and bake it a bit less, and end up with the same result? I don’t know. I have never thoroughly investigated this, and it might make a good project for another time (and another blog post).

Pan type

Is there some special significance to the Serious Eats recipe’s use of a springform pan instead of a regular pie dish? I don’t think there is. This appears to be a matter of convenience and preference. (I can imagine, in particular, that a springform pan would work better than a pie pan if all the pie pans you had were very shallow, and incapable of holding the requisite amount of crust and filling; whereas you could press the crust up the sides of the springform, using it as an impromptu pie dish.)

Crust baking time

It is difficult to analyze baking times, for several reasons.

First, many ovens are mis-calibrated. Set your oven to 350°F, and it may in fact heat to 325°F or 375°F. Oven thermometers can help (and I have two, myself), but only to an extent (as they tend to be imprecise as well as inaccurate—the latter fact demonstrable by buying two of them, using them side by side, and noting that they indicate different temperatures—which is right? who knows!). Ambient temperature may also vary within an oven (from bottom rack to middle to top; from left to right side; from front to back). Some ovens have a convection feature, while most do not; this further complicates the comparison.

Second, even given a known oven temperature, many factors may affect the time required to bring the crust (or any other baked dessert) to the desired state, not least of which is the composition and construction of the pan: is it steel, aluminum, glass, ceramic? Thick or thin? Nonstick or no? Dark or reflective? Deep or shallow? Each of these things can affect baking times. Other factors, such as the brand of butter (some butters have a higher fat content than others) and how the crust is pressed into the pan (loosely or densely), can also make a difference.

For these reasons, baking times for all sorts of desserts are often qualified with “… or until golden” (as in the case of all of the recipes we’re looking at today), or “… or until set”, “… or until browned and springs back when touched”, etc. In other words, the procedure is not “bake for X minutes”, but rather: “bake until the thing has reached state Y—however long that might take (but note that it’s likely to take in the neighborhood of X minutes)”. (Why give a time at all, then? To calibrate expectations; if a sponge cake should bake for an hour, you need not start checking it 5 minutes after baking begins; if a batch of cookies should bake for 10 minutes but yours have been baking for 25 and are still not done, then you know something is wrong; etc.)

The problem, of course, is that for someone who has never made a certain sort of recipe before, such qualitative descriptions are hard to judge. Just what exactly constitutes “golden”? (Must you have a bar of actual gold on hand, for visual comparison, or at least a PANTONE catalog? Should the list of essential kitchen tools for the aspiring dessert chef include a spectrophotometer?)

Prebaked crust.
(Is this “golden”? Was it not “golden” before…? Who knows…)
(click to enlarge)

The only real solution is to try it, do your best, see how it comes out, and adjust. Looking at pictures can help; more detailed descriptions can help; but there is really no substitute for experience, when it comes to judging these things. (That is the other reason explicit baking times are provided, along with qualitative descriptions—so that you have a sane default to fall back on, if you’re making the recipe for the first time, and judging whether or not the crust has turned ‘golden’ is too tricky or too unreliable.)

The bottom line is that the variation in baking times specified for the crust, among all of these recipes, is nearly meaningless. I will have to rely heavily on my own judgment and experience to know when the crust is done.

(Fortunately, the margin of error here is relatively large; both a slightly underbaked graham cracker crust and a slightly overbaked graham cracker crust taste almost the same as one which is baked exactly right. There is no sharp breakpoint, no sudden phase change. This is not true of all baked desserts—indeed, it’s not true of the filling for this very pie, which must be baked to a fairly specific consistency to taste ideal—but it’s true in the case of the crust. So if you’re not sure whether the thing is quite ‘golden’ enough, or just a bit too ‘golden’… don’t sweat it.)

Filling ingredients

(click to enlarge)

Lime juice

The filling is the heart and soul of a Key lime pie. Getting it just right is of paramount importance; so we must understand its composition—the ingredients, and how they go together—as thoroughly as possible.

Foremost among the questions before us is: must we use Key lime juice? Might regular¹¹ lime juice be sufficient? And as far as I can tell, the answer is clear: regular lime juice is just fine (and, perhaps, even better).

I am not alone in this judgment. Stella Parks of Serious Eats published an article titled “Key Limes? More Like Key LIES” that skewers the notion that only Key lime juice is acceptable for the pie of the same name; and the Key lime pie recipe from The Pioneer Woman (one of the ones I list in the “pie matrix” above) also uses regular, non-Key limes, with (reportedly) delicious results.

It so happens that Key limes are available near me… but they are very expensive, with a single Key lime costing about a dollar (whereas regular limes can be had at 3 for $1); when you recall that Key limes are much smaller than regular limes (perhaps one-third the volume), you can see that I’d be paying a whopping 800% premium. That seems excessive.

Bottled Key lime juice.
(click to enlarge)

Also available near me is bottled (from concentrate) Key lime juice, imported from Peru. Which of these things to use is a question I’ll tackle in the next section.

Sweetened condensed milk

The other key¹² ingredient in the filling is sweetened condensed milk.

Now, sweetened condensed milk is a delicious product in its own right¹³, and we should not look askance at any dessert recipe that uses it. However, it’s important to note that the reason it was originally used in Key lime pie is not choice, but necessity: as Wikipedia tells us, “Fresh milk was not a common commodity in the Florida Keys before modern refrigerated distribution methods.”

The reason it is still used in Key lime pie today—despite easy access to fresh milk—is a combination of tradition and the fact that Key lime pie recipes have, over the span of a century, been designed and optimized around the inclusion of this particular ingredient. So even though there’s no reason to believe that it’s impossible to substitute something else and still have the recipe work (or even work better), I will not even be considering any substitutions—at least, not before I’ve found a perfect Key lime pie recipe made in the usual manner.

As far as how much of it to use: as I explained earlier, most Key lime pie recipes call for 14 oz. of sweetened condensed milk, because—no matter the brand—it comes in 14-oz. cans.

… unless, of course, you buy a Russian brand of the stuff, in a Russian supermarket. (Sweetened condensed milk is very popular in Russia.) Then the amount you get in a can might vary as much as an ounce in either direction, and you may even find yourself buying one of these:

“Condensed cream with sugar”.
(click to enlarge)

This adorable mini-bucket holds 400 grams of delicious, cream-based sweetened condensed milk (which, the label informs us, came “straight from the farm!”). (14 oz. is equivalent to 396 grams.)

Will this degree of variation affect the recipe? Statistically, no. A difference of up to a half-ounce either way is indistinguishable from noise (‘noise’ in this case coming from many factors; e.g., the amount of product that remains in the can after you empty it—will you bother scraping out every last tiny drop? Probably not!).

Egg yolks

(click to enlarge)

The amount of variation between the recipes, in how many egg yolks to use in the filling, is distressing. For example, four of the recipes use 2, 3, 4, and 5 yolks respectively, despite the fact that each uses exactly identical amounts of both lime juice and sweetened condensed milk. What’s going on?

Might egg size be the answer? If we look at the egg sizes defined by the U.S. Department of Agriculture, we can see that “jumbo” eggs are almost 50% larger than “medium” eggs, and about 25% larger than “large” eggs (which Wikipedia tells us is the size most commonly referred to, for recipes).

Unfortunately, none of the recipes I looked at specify a size of egg from which to take the yolks. In any case, it does not seem that egg size disparities are sufficient to explain the degree of variation in yolk quantity from one recipe to another. So let’s ask: what are the yolks doing in this recipe?

Before we begin speculating, let’s see if someone already knows the answer. The Dessert Bible doesn’t include a Key lime pie recipe, but it does have a section on custard pies, which mentions that “eggs … add flavor and provide emulsification” (and Wikipedia confirms that egg yolk acts as an emulsifier). Might the egg yolks help to stabilize the mixture of lime juice and sweetened condensed milk, improving the texture? The Dessert Bible also tells us that in a lemon meringue pie recipe, increasing the quantity of yolks makes the filling “richer and smoother”. Egg yolks also contain a lot of fat, and fat affects how we perceive the flavor and texture of food (generally making it taste better, of course).

So there’s no clear answer (that I’ve been able to find), but based on available information, it seems likely that the egg yolks are critical to both the texture and the flavor of the pie filling. More egg yolks should yield a smoother, richer texture, and a deeper and more satisfying flavor—in short, should make the filling more custard-like.

Filling assembly

Beating the yolks incorporates air into them, and makes the mixture more smooth. In many other sorts of recipes that include egg yolks, beating the yolks yields a fluffier, creamier final product (whereas not beating the yolks makes for a denser final product). The effect is unlikely to be large, however, because the yolks are mixed with other dense liquids, and no flour; thus no stable foam can form and solidify. (There’s that background knowledge again!)

Baking temperature

The mean, median, and mode of the baking temperatures across the sampled recipes is 350°F. There does not seem to be a clear reason for either of the two outliers.

Baking time

At first glance, there seems to be a great deal of variation in the specified baking time. But a closer look reveals that the low outlier—Allrecipes.com recipe (b)—is also the recipe that includes no egg yolks, way more sweetened condensed milk than all the others, and is unique in including sour cream. Clearly, something different is going on in this recipe; to evaluate the others, we have to discard this one.

The variation in the remaining recipes doesn’t seem to be correlated with any obvious property of the filling. It’s entirely possible that variations in oven performance suffices to explain it. In any case, most of the recipes quite correctly caution you to bake not for any set amount of time, but “until the filling is just set”.

But what does that look like? What is “just set”? Once again we can turn to the Dessert Bible for a hint, which (in the sections on custard pies and cheesecakes—both of which are, in their own way, similar to Key lime pie) suggests that we should bake the dessert until the filling is no longer liquid, and is clearly fully set on the edges, but jiggles a bit in the middle, when shaken.

Topping ingredients

Heavy cream, granulated white sugar, and pure vanilla extract.
(click to enlarge)

I have little to say here; whipped cream is a pretty simple thing. Analyses of its properties, techniques, etc., are ubiquitous. Assuming that our basic technique is sound, variation in preferences will almost certainly outweigh most other considerations when it comes to nailing down this part of the recipe.

The one thing to note is that the use of confectioner’s (rather than granulated) sugar is almost certainly for the purpose of stabilizing the whipped cream (i.e., preventing it from “weeping” or “separating”). (Note that the inclusion of mascarpone or cream cheese actually serves a similar function, though it has a larger effect on the flavor and texture of the final product.)

Topping process

Making whipped cream.
(click to enlarge)

Once again, making whipped cream is a simple process. There are some tips and tricks one can apply; these may be googled easily enough. “How to make the perfect whipped cream” is certainly beyond the scope of this post, and is well-trod ground in any case.

Post-processing

Chilling the product after baking is a step common to many recipes for baked desserts that contain eggs and dairy (one finds it often enough in recipes for cheesecakes, for brownies, etc.). What does it accomplish? Empirically: it improves the flavor. This fact is asserted by many cookbook writers, food bloggers, etc.; it can be verified easily enough in practice. In my experience, it’s 100% true. Brownies, cheesecakes, and many other such desserts just don’t taste as good immediately after baking as they do after sitting in the fridge for a good few hours. (Why is this? I don’t know. It’s an excellent question, which I’ll have to put off answering for another time.)

So, if all but one of the recipes I surveyed include an instruction to chill the baked pie, then it seems entirely plausible that Key lime pie is simply another one of the many desserts for which the flavor is improved after several hours in the fridge.

Synthesis (improvisation)

This part is where preference and opinion enter the equation.

For some parts of the recipe, the analysis we did in the previous section will point to a clear best choice among the available option; for others, there may really be only one way to go. But in some cases, there will be multiple possibilities, and selecting among them will be a matter of taste. Well, what’s life without strong opinions, eh?

Crust

The pie dish I intend to use is a particularly deep (the better to hold a lot of pie filling) 9″ model. So I’m going to err on the generous side, and use 11 graham crackers. Correspondingly, I’m going to go with 5 tbsp of both butter and sugar; and, as per the Dessert Bible recipe, it’ll be dark brown sugar (the flavor will be more complex that way). I see no compelling reason to add salt, despite the fact that I use unsalted butter.¹⁴

I will prebake the pie crust at 350°F, once again due to lack of any good reason to deviate from this standard temperature (and to avoid having to wait for the oven temperature to adjust between prebaking the crust and baking the filling). As for baking time, I’ll default to 10 minutes, at which point I’ll start checking the crust to make sure it’s properly “golden”.

Filling

Limes

Here I digress to recount a curious thing.

I didn’t pay close attention when purchasing the limes for this pie, but after coming home and unpacking my groceries, I noticed something about the limes I’d bought:

(click to enlarge)

I got these four limes from the same local greengrocer; they were all heaped together, and not labeled any differently. And yet it’s clear that what I’ve got here is two different varieties of lime! The two limes on the left (what I’ve labeled #s 1 and 2) have a lighter green and smoother rind, and are more spherical; while the two limes on the right (#s 3 and 4) have a much darker and knobbier rind, and are more elongated.

This difference was confirmed after I zested the limes and quartered them (in order to squeeze out the juice). Here they are, in cross-section:

Lime #2 (top) and lime #3 (bottom)
(click to enlarge)

(Lime #1 was similar to #2, and #4 to #3, inside as well as out.)

But do these two varieties of lime differ in any important way—anything other than shape and texture and so on? They do indeed, which I discovered when I began squeezing the juice from them. Limes #1 and #2 (the lighter, smoother, more spherical ones) produced a large amount of juice. But limes #3 and #4 (the darker, knobbier, more elongated ones) were extremely dry. I do not exaggerate even a little bit when I say that I got only one drop of juice from the entirety of lime #3. (Lime #4 was slightly juicier—but not by much.)

My point? That you can (as I did) buy a bunch of perfectly ordinary limes—intending to zest and then juice them—and then find that while you’ve ended up with plenty of zest to garnish the pie with, you haven’t nearly enough juice for the filling.

What to do? Well, here’s what I did: I simply supplemented the juice I managed to squeeze out of the limes with some bottled from-concentrate Key lime juice. There’s no law against mixing-and-matching!

Of course, that makes pinning down the ideal amount of juice tricky. Key limes, as we know, are more acidic than regular limes; so if using regular lime juice, we should probably use more of it (to give the filling a properly strong lime flavor). It seems to me that 1⁄2 cup (the most common amount in the recipes I surveyed) is sensible if using Key lime juice; but if using regular lime juice, the amount should be increased to 2⁄3 or even 3⁄4 cup (if you prefer a really strong flavor, and enjoy sour sweets). If using a roughly equal mix, as I did, 2⁄3 cup seems like a sensible middle ground.

Should we use lime zest? I don’t see why not; it should enhance the flavor even more (due to the oils in the zest). One reason not to use it would be if you dislike the resulting texture; if you want your filling to be totally smooth, skip the zest. (A compromise might be to mince the zest before adding it; this’ll make it less noticeable in the filling.) If using zest, 1 tbsp should do the trick (this should approximately equal the zest of one lime).

Other ingredients

The sweetened condensed milk is a no-brainer: use one 14-oz. can of it. (Remember, buy the good stuff! No substitutes!)

That leaves the egg yolks. As we saw in our analysis above, the recipes really vary quite a bit; and more yolks should yield a smoother, richer, texture, and a deeper, more satisfying flavor, making the filling more custard-like. This is exactly what I want, so I’m going to err on the side of more yolks. How many, exactly? Well, the high end of the spread is 5 egg yolks; that recipe doesn’t specify egg size, but the most common egg size in the U.S. is large. I, on the other hand, usually buy extra-large eggs, which are 12% larger in volume. I am therefore going to use the yolks from four extra-large eggs.

Assembly

Do I want to beat the egg yolks before adding the other ingredients? I think I do. I don’t want the filling to be very dense; it should be a bit fluffy, and as smooth as possible.

Baking temperature & time

For the baking temperature, I’m going to stick with the clear standard here: 350°F. As for baking time—well, certainly we should expect that it’ll take about 10 minutes on the low end; at that point, it would be sensible to start checking the pie every couple of minutes, to make sure not to overbake it. It should be removed from the oven when just set (as we discussed in the analysis section).

Topping

As mentioned before, whipped cream is a simple thing. I’m not going to get fancy by adding mascarpone or any such thing. I will, however, use vanilla extract in the topping, because vanilla tremendously enhances the flavor of whipped cream. (If you do use vanilla extract, make sure it’s the real stuff! No “imitation vanilla extract”, no synthetics, etc.)

Post-processing

Chilling the pie after baking seems like a solid bet. I see no reason to skip this step.

Recipe

And here it is: the Key lime pie recipe I promised at the beginning of this post.

Equipment

• food processor
• small saucepan (or something else in which to melt butter)
• 9″ pie dish
• oven
• Microplane zester (or good box grater)
• mixing bowls (one medium, one large)
• measuring spoons
• liquid measuring cup
• electric hand mixer (or stand mixer; or, in a pinch, a good whisk and a strong arm)
• miscellaneous kitchen tools (knives, cutting board, etc.)

Ingredients

Note: All ingredients are listed in the order in which they’re to be added.

Crust

(click to enlarge)

• 11 graham crackers
• 5 tbsp dark brown sugar
• 5 tbsp butter

Filling

(click to enlarge)

• 4 extra-large egg yolks
• 2⁄3 cup lime juice
  Alternatively: 1⁄2 cup Key lime juice
• 14 oz (i.e., 1 can) sweetened condensed milk
• 1 tbsp lime zest, minced

Topping

(click to enlarge)

• 2 cups heavy cream
• 1⁄3 cup granulated sugar
• 2 tsp vanilla extract

• 1 tbsp lime zest (as garnish)

Instructions

Flowchart of the process.
(click to enlarge)

Part I: Prepare the crust

1. Melt the 5 tbsp butter on low heat; remove from heat and set aside.
2. In a food processor, process the 11 graham crackers with the 5 tbsp dark brown sugar into fine crumbs.
3. Add the melted butter to the food processor; process until fully incorporated.
4. Turn out the crust mixture into a 9″ pie dish. Press the crust into bottom and sides of pie dish.
5. Place crust in refrigerator to chill for approximately 15 minutes. Set oven to 350°F, to preheat.

Part II: Begin preparing the filling

While the crust is chilling and the oven preheating, start working on the filling:

6. Zest the limes.
7. Halve (or quarter) and juice the limes. Make sure you have the right amount of juice (2⁄3 cup if using regular lime juice, or 1⁄2 cup if using Key lime juice).
8. Separate the eggs; place the (four extra-large) yolks into medium mixing bowl. (Preserve the whites for another use.)
9. Open the can of sweetened condensed milk.

Part III: Prebake the crust

10. Remove crust from refrigerator (where it has now been chilling for about 15 minutes), and put it in the oven. Bake for 10 minutes (or until golden).

Part IV: Mix the filling

While the crust is baking, prepare the filling mixture:

11. Using the hand mixer with the whisk attachment(s), beat the 4 extra-large egg yolks on high speed until light and thick.
12. Add the 1 tbsp lime zest to egg yolks, and beat on low speed until incorporate.
13. Add the 14 oz. sweetened condensed milk and 2⁄3 cup lime juice; beat on low speed until just incorporated.

Part V: Fill and bake the pie

14. Remove the crust from the oven once it’s done prebaking. (At this point, the filling mixture should be done.)
15. Pour filling mixture into prebaked pie crust.
16. Put filled pie back in the oven; bake for 10 minutes, or until just set (it should jiggle slightly in the middle, while the edges should be fully set).
17. Remove pie from oven, and set on rack to cool.

Part VI: Make the topping

Make whipped cream. Use your preferred method, or as follows:

18. Combine the 2 cups heavy cream, 1⁄3 cup white sugar, and 2 tsp vanilla extract in a large bowl.
19. With electric hand mixer, beat cream on high speed until stiff peaks form.

Part VII: Assemble the pie

20. Spread whipped cream over the cooled pie.
21. Garnish with lime zest.

Part VIII: Chill and enjoy!

22. Chill pie in the fridge for at least 4 hours (preferably overnight).
23. Eat the pie!

What can go wrong

The Dessert Bible includes a “What Can Go Wrong” section after many of the recipes therein; I have always found these to be tremendously useful.

Fortunately for me (and for anyone reading this and thinking of trying the recipe), certain readers of this blog have already tried their hand at making the pie in accordance with the instructions above. I am given to understand that the results were delicious—but flawed. (Don’t get discouraged, anonymous blog readers! Keep trying! The world is better for your efforts!) Here, then are some things that can go wrong when making a Cognitive Pie™ Key lime pie.

Inadequate egg separation

Perfectly separating yolks from whites is tricky. Making sure no yolk gets into the whites (critical if you’re making meringue, for instance) takes care but is doable; making sure no white gets into the yolks is impossible. Still, it’s very possible to minimize how much egg white ends up in your yolks—and it’s definitely something you want to aim for, in recipes like this.

This is good:

(click to enlarge)

This is not quite as good:

(click to enlarge)

What happens if whites get into the yolks? The texture suffers. It’ll be a bit more rubbery and less smooth—not ruinous, but not perfect. To avoid this, take care when separating the eggs. One trick that helps here is to make sure the eggs are cold (i.e., right out of the fridge) when you separate them.

Not beating the egg yolks well enough

This is step #11 in the recipe. You want the yolks to be very thick and pale before you add the other ingredients. (If you try to beat the yolks after you’ve already added the lime juice or sweetened condensed milk, it won’t work.)

If you don’t do this quite well enough, it’s not the end of the world… but the filling will be denser, and not as smooth and “custardy”, as it could be.

Overbeating the filling

Counterintuitive though it might be, given the preceding heading, it’s true: you can overbeat Key lime pie filling. The trick is, after you’ve added the juice, zest, and sweetened condensed milk to the egg yolks—then you want to minimize how much you beat them. Mix just to incorporate fully—no more.

Otherwise, this can happen:

(click to enlarge)

Yes, the filling bubbles and cracks. You want the filling to be smooth and without any bubbles or cracks when it’s baked. So be gentle with that filling, once you’ve added things to the beaten yolks!

Conclusion

We started with a general approach to developing procedural knowledge: the “Saturation, Distillation, Improvisation” (or ‘SDI’) method.

Building on the SDI concept, we refined it into a procedure for developing recipes, which goes like this:

1. Decide on a type of dish to develop a recipe for.
2. Gather a handful (perhaps a half-dozen or so) recipes from various sources (via casual googling, perhaps).
3. Build a table that provides a quick, at-a-glance comparison of the key parts of each recipe (both the ingredients and all relevant elements of the process).
4. Identify commonalities and differences between the recipes. Note ranges of variation. Identify clear outliers.
5. Examine each dimension of variation, aiming to understand the reason behind each similarity and each range of variation. Try to understand the role that each part (each ingredient and each step in the procedure) of the recipe plays.
6. Go through each element of the recipe again, choosing a point along the dimension of variation. Make use of your background knowledge, your understanding of relevant situational factors, and your personal preferences.
7. Assemble the synthesized elements into a complete recipe.
8. Try out the recipe; evaluate results; iterate. (This is not part of the technique per se, but is a crucial follow-up step in practice.)

We then applied this procedure to the task of developing an original recipe for a popular dessert: Key lime pie.

Finally, recipe in hand, we (well, I) baked the pie. The result was delicious!

Slice of Key lime pie.
(click to enlarge)