Why corporate penny-shaving backfires. (Also, how to do a layoff right.)

One of the clearest signs of corporate decline (2010s Corporate America is like 1980s Soviet Russia, in terms of its low morale and lethal overextension) is the number of “innovations” that are just mean-spirited, and seem like prudent cost-cutting but actually do minimal good (and, often, much harm) to the business.

One of these is the practice of pooling vacation and sick leave in a single bucket, “PTO”. Ideally, companies shouldn’t limit vacation or sick time at all– but my experience has shown “unlimited vacation” to correlate with a negative culture. (If I ran a company, it would institute a mandatory vacation policy: four weeks minimum, at least two of those contiguous.) Vacation guidelines need to be set for the same reason that speed limits (even if intentionally under-posted, with moderate violation in mind) need to be there; without them, speed variance would be higher on both ends. So, I’ve accepted the need for vacation “limits”, at least as soft policies; but employers expect their people to either use a vacation day for sick leave, or come into the office while sick, are just being fucking assholes.

These PTO policies are, in my view, reckless and irresponsible. They represent a gamble with employee health that I (as a person with a manageable but irritating disability) find morally repugnant. It’s bad enough to deny rest to someone just because a useless bean-counter wants to save the few hundred dollars paid out for unused vacation when someone leaves the company. But by encouraging the entire workforce to show up while sick and contagious, they subject the otherwise healthy to an unnecessary germ load. Companies with these pooled leave, “PTO”, policies end up with an incredibly sickly workforce. One cold just rolls right into another, and the entire month of February is a haze of snot, coughing, and bad code being committed because half the people at any given time are hopped up on cold meds and really ought to be in bed. It’s not supposed to be this way. This will shock those who suffer in open-plan offices, but an average adult is only supposed to get 2-3 colds per year, not the 4-5 that are normal in an open-plan office (another mean-spirited tech-company “innovation”) or the 7-10 per year that is typical in pooled-leave companies.

The math shows that PTO policies are a raw deal even for the employer. In a decently-run company with an honor-system sick leave policy, an average healthy adult might have to take 5 days off due to illness per year. (I miss, despite my health problems, fewer than that.) Under PTO, people push themselves to come in and only stay home if they’re really sick. Let’s say that they’re now getting 8 colds per year instead of the average 2. (That’s not an unreasonable assumption, for a PTO shop.) Only 2 or 3 days are called-off, but there are a good 24-32 days in which the employee is functioning below 50 percent efficiency. Then there are the morale issues, and the general perception that employees will form of the company as a sickly, lethargic place; and the (mostly unintentional) collective discovery of how low a level of performance will be tolerated. January’s no longer about skiing on the weekends and making big plans and enjoying the long golden hour… while working hard, because one is refreshed. It’s the new August; fucking nothing gets done because even though everyone’s in the office, they’re all fucking sick with that one-rolls-into-another months-long cold. That’s what PTO policies bring: a polar vortex of sick.

Why, if they’re so awful, do companies use them? Because HR departments often justify their existence by externalizing costs elsewhere in the company, and claiming they saved money. So-called “performance improvement plans” (PIPs) are a prime example of this. The purpose of the PIP is not to improve the employee. Saving the employee would require humiliating the manager, and very few people have the courage to break rank like that. Once the PIP is written, the employee’s reputation is ruined, making mobility or promotion impossible. The employee is stuck in a war with his manager (and, possibly, team) that he will almost certainly lose, but he can make others lose along the way. To the company, a four-month severance package is far cheaper than the risk that comes along with having a “walking dead” employee, pissing all over morale and possibly sabotaging the business, in the office for a month. So why do PIPs, which don’t even work for their designed intention (legal risk mitigation) unless designed and implemented by extremely astute legal counsel, remain common? Well, PIPs a loss to the company, even compared to “gold-plated” severance plans. We’ve established that. But they allow the HR department to claim that it “saved money” on severance payments (a relatively small operational cost, except when top executives are involved) while the costs are externalized to the manager and team that must deal with a now-toxic (and if already toxic before the PIP, now overtly destructive) employee. PTO policies work the same way. The office becomes lethargic, miserable, and sickly, but HR can point to the few hundred dollars saved on vacation payouts and call it a win.

On that, it’s worth noting that these pooled-leave policies aren’t actually about sick employees. People between the ages of 25 and 50 don’t get sick that often, and companies don’t care about that small loss. However, their children, and their parents, are more likely to get sick. PTO policies aren’t put in place to punish young people for getting colds. They’re there to deter people with kids, people with chronic health problems, and people with sick parents from taking the job. Like open-plan offices and the anxiety-inducing micromanagement often given the name of “Agile”, it’s back-door age and disability discrimination. The company that institutes a PTO policy doesn’t care about a stray cold; but it doesn’t want to hire someone with a special-needs child. Even if the latter is an absolute rock star, the HR department can justify itself by saying it helped the company dodge a bullet.

Let’s talk about cost cutting more generally, because I’m smarter than 99.99% of the fuckers who run companies in this world and I have something important to say.

Companies don’t fail because they spend too much money. “It ran out of money” is the proximate cause, not the ultimate one. Some fail when they cease to excel and inspire (but others continue beyond that point). Some fail, when they are small, because of bad luck. Mostly, though, they fail because of complexity: rules that don’t make sense and block useful work from being done, power relationships that turn toxic and, yes, recurring commitments and expenses that can’t be afforded (and must be cut). Cutting complexity rather than cost should be the end goal, however. I like to live with few possessions not because I can’t afford to spend the money (I can) but because I don’t want to deal with the complexity that they will inject into my life. It’s the same with business. Uncontrolled complexity will cause uncontrolled costs and ultimately bring about a company’s demise. What does this mean about cutting costs, which MBAs love to do? Sometimes it’s great to cut costs. Who doesn’t like cutting “waste”? The problem there is that there actually isn’t much obvious waste to be cut, so after that, one has to focus and decide on which elements of complexity are unneeded, with the understanding that, yes, some people will be hurt and upset. Do we need to compete in 25 businesses, when we’re only viable in two? This will also cut costs (and, sadly, often jobs).

The problem, see, is that most of the corporate penny-shaving increases complexity. A few dollars are saved, but at the cost of irritation and lethargy and confusion. People waste time working around new rules intended to save trivial amounts of money. The worst is when a company cuts staff but refuses to reduce its internal complexity. This requires a smaller team to do more work– often, unfamiliar work that they’re not especially good at or keen on doing; people were well-matched to tasks before the shuffle, but that balance has gone away. The career incoherencies and personality conflicts that emerge are… one form of complexity.

The problem is that most corporate executives are “seagull bosses” (swoop, poop, and fly away) who see their companies and jobs in a simple way: cut costs. (Increasing revenue is also a strategy, but that’s really hard in comparison.) A year later, the company is still failing not because it failed to cut enough costs or people, but because it never did anything about the junk complexity that was destroying it in the first place.

Let’s talk about layoffs. The growth of complexity is often exponential, and firms inevitably get to a place where they are too complex (and, a symptom of this is that operations are too expensive) to survive. The result is that it needs to lay people off. Now, layoffs suck. They really fucking do. But there’s a right way and a wrong way to execute one. To do a layoff right, the company needs to cut complexity and cut people. (Otherwise, it will have more complexity per capita, the best people will get fed up and leave, and the death spiral begins.) It also needs to cut the right complexity; all the stuff that isn’t useful.

Ideally, the cutting of people and cutting of complexity would be tied together. Unnecessary business units being cut usually means that people staffed on them are the ones let go. The problem is that that’s not very fair, because it means that good people, who just happened to be in the wrong place, will lose their jobs. (I’d argue that one should solve this by offering generous severance, but we already know why that isn’t a popular option, though it should be.) The result is that when people see their business area coming into question, they get political. Of course this software company needs a basket-weaving division! In-fighting begins. Tempers flare. From the top, the water gets very muddy and it’s impossible to see what the company really looks like, because everyone’s feeding biased information to the executives. (I’m assuming that the executive who must implement the cuts is acting in good faith, which is not always true.) What this means is that the crucial decision– what business complexity are we going to do without?– can’t be subject to a discussion. Debate won’t work. It will just get word out that job cuts are coming, and political behavior will result. The horrible, iron fact is that this calls for temporary autocracy. The leader must make that call in one fell swoop. No second guessing, no looking back. This is the change we need to make in order to survive. Good people will be let go, and it really sucks. However, seeing as it’s impossible to execute a large-scale layoff without getting rid of some good people, I think the adult thing to do is write generous severance packages.

Cutting complexity is hard. It requires a lot of thought. Given that the information must be gathered by the chief executive without tipping anyone off, and that complex organisms are (by definition) hard to factor, it’s really hard to get the cuts right. Since the decision must be made on imperfect information, it’s a given that it usually won’t be the optimal cut. It just has to be good enough (that is, removing enough complexity with minimal harm to revenue or operations) that the company is in better health.

Cutting people, on the other hand, is much easier. You just tell them that they don’t have jobs anymore. Some don’t deserve it, some cry, some sue, and some blog about it but, on the whole, it’s not actually the hard part of the job. This provides, as an appealing but destructive option, the lazy layoff. In a lazy layoff, the business cuts people but doesn’t cut complexity. It just expects more work from everyone. All departments lose a few people! All “survivors” now have to do the work of their fallen brethren! The too-much-complexity problem, the issue that got us to the layoff in the first place… will figure itself out. (It never does.)

Stack ranking is a magical, horrible solution to the problem. What if one could do a lazy layoff but always cull the “worst” people? After all, some people are of negative value, especially considering the complexity load (in personality conflicts, shoddy work) they induce. The miracle of stack ranking is that it turns a layoff– otherwise, a hard decision guaranteed to put some good people out of work– into an SQL query. SELECT name FROM Employee WHERE perf <= 3.2. Since the soothsaying of stack ranking has already declared the people let-go as bottom-X-percent performers, there’s no remorse in culling them. They were dead weight”. Over time, stack ranking evolves into a rolling, continuous lazy layoff that happens periodically (“rank-and-yank”).

It’s also dishonest. There are an ungodly number of large technology companies (over 1,000) that claim to have “never had a layoff”. That just isn’t fucking true. Even if the CEO was Jesus Christ himself, he’d have to lay people off because that’s just how business works. Tech-company sleazes just refuse to use the word “layoff”, for fear of losing their “always expanding, always looking for the best talent!” image. So they call it a “low performer initiative” (stack ranking, PIPs, eventual firings). What a “low-performer initiative” (or stack ranking, which is a chronic LPI) inevitably devolves into is a witch hunt that turns the organization into pure House of Cards politics. Yes, most companies have about 10 percent who are incompetent or toxic or terminally mediocre and should be sent out the door. Figuring which 10 percent those people are, is not easy. People who are truly toxic generally have several years’ worth of experience drawing a salary without doing anything, and that’s a skill that improves with time. They’re really good at sucking (and not getting caught). They’re adept political players. They’ve had to be; the alternative would have been to have grown a work ethic. Most of what we as humans define as social acceptability is our ethical immune system, which can catch and punish the small-fry offenders but can’t do a thing about the cancer cells (psychopaths, parasites) that have evolved to the point of being able to evade or even redirect that rejection impulse. The question of how to get that toxic 10 percent out is an unsolved one, and I don’t have space to tackle it now, but the answer is definitely not stack ranking, which will always clobber several unlucky good-faith employees for every genuine problem employee it roots out.

Moreover, stack ranking has negative permanent effects. Even when not tied to a hard firing percentage, its major business purpose is still to identify the bottom X percent, should a lazy layoff be needed. It’s a reasonable bet that unless things really go to shit, X will be 5 or 10 or maybe 20– but not 50. So stack ranking is really about the bottom. The difference between the 25th percentile and 95th percentile, in stack ranking, really shouldn’t matter. Don’t get me wrong: a 95th-percentile worker is often highly valuable and should be rewarded. I just don’t have any faith in the ability of stack ranking to detect her, just as I know some incredibly smart people who got mediocre SAT scores. Stack ranking is all about putting people at the bottom, not the top. (Top performers don’t need it and don’t get anything from it.)

The danger of garbage data (and, #YesAllData generated by stack ranking is garbage) is that people tend to use it as if it were truth. The 25th-percentile employee isn’t bad enough to get fired… but no one will take him for a transfer, because the “objective” record says he’s a slacker. The result of this– in conjunction with closed allocation, which is already a bad starting point– is permanent internal immobility. People with mediocre reviews can’t transfer because the manager of the target team would prefer a new hire (with no political strings attached) over a sub-50th-percentile internal. People with great reviews don’t transfer for fear of upsetting the gravy train of bonuses, promotions, and managerial favoritism. Team assignments become permanent, and people divide into warring tribes instead of collaborating. This total immobility also makes it impossible to do a layoff the right way (cutting complexity) because people develop extreme attachments to projects and policies that, if they were mobile and therefore disinterested, they’d realize ought to be cut. It becomes politically intractable to do the right thing, or even for the CEO to figure out what the right thing is. I’d argue, in fact, that performance reviews shouldn’t be part of a transfer packet at all. The added use of questionable, politically-laced “information” is just not worth the toxicity of putting that into policy.

A company with a warring-departments dynamic might seem like a streamlined, efficient, and (most importantly) less complex company. It doesn’t have the promiscuous social graph you might expect to see in an open allocation company. People know where they are, who they report to, and who their friends and enemies are. The problem, with this insight, is that there’s hot complexity and cold complexity. Cold complexity is passive and occasionally annoying, like a law from 1890 that doesn’t make sense and is effectively never enforced. When people collaborate “too much” and the social graph of the company seems to have “too many” edges, there’s some cold complexity there. It’s generally not harmful. Open allocation tends to generate some cold complexity. Rather than metastasize into an existential threat to the company, it will fade out of existence over time. Hot complexity, which usually occurs in an adversarial context, is a kind that generates more complexity. Its high temperature means there will be more entropy in the system. Example: a conflict (heat) emerges. That, alone, makes the social graph more complex because there are more edges of negativity. Systems and rules are put in place to try to resolve it, but those tend to have two effects. First, they bring more people (those who had no role in the initial conflict, but are affected by the rules) into the fights. Second, the conflicting needs or desires of the adversarial parties are rarely addressed, so both sides just game the new system, which creates more complexity (and more rules). Negativity and internal competition create the hot complexity that can ruin a company more quickly than an executive (even if acting with the best intentions) can address it.

Finally, one thing worth noting is the Welch Effect (named for Jack Welch, the inventor of stack-ranking). It’s one of my favorite topics because it has actually affected me. The Welch Effect pertains to the fact that when a broad-based layoff occurs, the people most likely to be let go aren’t the worst (or best) performers, but newest members of macroscopically underperforming teams. Layoffs (and stack ranking) generally propagate down the hierarchy. Upper management disburses bonuses, raises, and layoff quotas based on the macroscopic performance of the departments under it, and at each level, the node operators (managers) slice the numbers based on how well they think each suborganization did (plus or minus various political modifiers). At the middle-management layer, one level separated from the non-managerial “leaves”, it’s the worst-performing teams that have to vote the most people off the island. It tends to be those most recently hired who get the axe. This isn’t especially unfair or wrong, for that middle manager; there’s often no better way to do it than to strike the least-embedded, least-invested junior hire.

The end result of the Welch Effect, however, is that the people let go are often those who had the least to do with their team’s underperformance. (It may be a weak team, it may be a good team with a bad manager, or it may be an unlucky team.) They weren’t even there for very long! It doesn’t cause the firm to lay off good people, but it doesn’t help it lay off bad people either. It has roughly the same effect as a purely seniority-based layoff, for the company as a whole. Random new joiners are the ones who are shown out the door. It’s bad to lose them, but it rarely costs the company critical personnel. Its effect on that team is more visibly negative: teams that lose a lot of people during layoffs get a public stink about them, and people lose the interest in joining or even helping them– who wants to work for, or even assist, a manager who can’t protect his people?– so the underperforming team becomes even more underperforming. There are also morale issues with the Welch Effect. When people who recently joined lose their jobs (especially if they’re fired “for performance” without a severance) it makes the company seem unfair, random, and capricious. The ones let go were the ones who never had the chance to prove themselves. In a one-off layoff, this isn’t so destructive. The Welch Effected usually move on to better jobs anyway. However, when a company lays off in many small cuts, or disguises a layoff as a “low-performer initiative”, the Welch Effect firings demolish belief in meritocracy.

That, right there, explains why I get so much flak over how I left Google. Technically, I wasn’t fired. But I had a disliked, underdelivering manager who couldn’t get calibration points for his people (a macroscopic issue that I had nothing to do with) and I was the newest on the team, so I got a bad score (despite being promised a reasonable one– a respectable 3.4, if it matters– by that manager). Classic Welch Effect. I left. After I was gone I “leaked” the existence of stack ranking within Google. I wasn’t the first to mention that it existed there, but I publicized it enough to become the (unintentional) slayer of Google Exceptionalism and, to a number of people I’ve never met and to whom I’ve never done any wrong, Public Enemy #1. I was a prominent (and, after things went bad, fairly obnoxious) Welch Effectee, and my willingness to share my experience changed Google’s image forever. It’s not a disliked company (nor should it be) but its exceptionalism is gone. Should I have done all that? Probably not. Is Google a horrible company? No. It’s above average for the software industry (which is not an endorsement, but not damnation either.) Also, my experiences are three years old at this point, so don’t take them too seriously. As of November 2011, Google had stack ranking and closed allocation. It may have abolished those practices and, if it has, then I’d strongly recommend it as a place to work. It has some brilliant people and I respect them immensely.

In an ideal world, there would be no layoffs or contractions. In the real world, layoffs have to happen, and it’s best to do them honestly (i.e. don’t shit on departing employees’ reputations by calling it a “low performer initiative”). As with more minor forms of cost-cutting (e.g. new policies encouraging frugality) it can only be done if complexity (that being the cause of the organization’s underperformance) is reduced as well. That is the only kind of corporate change that can reverse underperformance: complexity reduction.

If complexity reduction is the only way out, then why is it so rare? Why do companies so willingly create personnel and regulatory complexity just to shave pennies off their expenses? I’m going to draw from my (very novice) Buddhist understanding to answer this one. When the clutter is cleared away… what is left? Phrases used to define it (“sky-like nature of the mind”) only explain it well to people who’ve experienced it. Just trust me that there is a state of consciousness that can be attained when gross thoughts are swept away, leaving something more pure and primal. Its clarity can be terrifying, especially the first time it is experienced. I really exist. I’m not just a cloud of emotions and thoughts and meat. (I won’t get into death and reincarnation and nirvana here. That goes farther than I need, for now. Qualia, or existence itself, as opposed my body hosting some sort of philosophical zombie, is both miraculous and the only miracle I actually believe in.) Clarity. Essence. Those are the things you risk encountering with simplicity. That’s a good thing, but it’s scary. There is a weird, paradoxical thing called “relaxation-induced anxiety” that can pop up here. I’ve fought it (and had some nasty motherfuckers of panic attacks) and won and I’m better for my battles, but none of this is easy.

So much of what keeps people mired in their obsessions and addictions and petty contests is an aversion to confronting what they really are, a journey that might harrow them into excellence. I am actually going to age and die. Death can happen at any time, and almost certainly it will feel “too soon”. I have to do something, now, that really fucking matters. This minute counts, because I may not get another in this life. People are actually addicted to their petty anxieties that distract them from the deeper but simpler questions. If you remove all the clutter on the worktable, you have to actually look at the table itself, and you have to confront the ambitions that impelled you to buy it, the projects you imagined yourself using it for (but that you never got around to). This, for many people, is really fucking hard. It’s emotionally difficult to look at the table and confront what one didn’t achieve, and it’s so much easier to just leave the clutter around (and to blame it).

Successful simplicity leads to, “What now?” The workbench is clear; what are we going to do with it? For an organization, such simplicity risks forcing it to contend with the matter of its purpose, and the question of whether it is excelling (and, relatedly, whether it should). That’s a hard thing to do for one person. It’s astronomically more difficult for a group of people with opposing interests, and among whom excellence is sure to be a dirty word (there are always powerful people who prefer rent-seeking complacency). It’s not surprising, then, that most corporate executives say “fuck it” on the excellence question and, instead, decide it suffices to earn their keep to squeeze employees with mindless cost-cutting policies: pooled sick leave and vacation, “employee contributions” on health plans, and other hot messes that just ruin everything. It feels like something is getting done, though. Useless complexity is, in that way, existentially anxiolytic and addictive. That’s why it’s so hard to kill. But it, if allowed to live, will kill. It can enervate a person into decoherence and inaction, and it will reduce a company to a pile of legacy complexity generated by self-serving agents (mostly, executives). Then it falls under the MacLeod-Gervais-Rao-Church theory of the nihilistic corporation; the political whirlpool that remains once an organization has lost its purpose for existing.

At 4528 words, I’ve said enough.

Silicon Valley and the Rise of the Disneypreneur

Someone once explained the Las Vegas gambling complex as “Disneyland for adults”, and the metaphor makes a fair amount of sense. The place sells a fantasy– expensive shows, garish hotels (often cheap or free if “comped”) and general luxury– and this suspension of reality enables people to take financial risks they’d usually avoid, giving the casino an edge. Comparing Silicon Valley to Vegas, also, makes a lot of sense. Even more than a Wall Street trading floor, it’s casino capitalism. Shall we search for some kind of transitivity? Yes, indeed. Is it possible that Silicon Valley is a sort of “Disneyland”? I think so.

It starts with Stanford and Palo Alto. The roads are lined with palm trees that do not grow there naturally, and cost tens of thousands of dollars a piece to plant. The whole landscape is designed and fake. In a clumsy attempt to lift terminology from Southern aristocrats, Stanford’s nickname is “the Farm”. At Harvard or Princeton, there’s a certain sense of noblesse oblige that students are expected to carry with them. A number of Ivy Leaguers eschew investment banking in favor of a program like Teach for America. Not so much at Stanford, which has never tempered itself with Edwardian gravity (by, for example, encouraging students to read literature from civilizations that have since died out) in the way that East Coast and Midwestern colleges have. The rallying cry is, “Go raise VC.” Then, they enter a net of pipelines: Stanford undergrad to startup, startup to EIR gig, EIR to founder, founder to venture capitalist. The miraculous thing about is that progress on this “entrepreneurial” path is assured. One never needs to take any risk to do it! Start in the right place, don’t offend the bosses-I-mean-investors, and there are three options: succeed, fail up, or fail diagonal-up. Since they live in an artificial world in which real loss isn’t possible for them, but one that also limits them from true innovation, they perform a sort of Disney-fied entrepreneurship. They’re the Disneypreneurs.

Just as private-sector bureaucrats (corporate executives) who love to call themselves “job creators” (and who only seem to agree on anything when they’re doing the opposite) are anything but entrepreneurs, I tend to think of these kids as not real entrepreneurs. Well, because I’m right, I should say it more forcefully. They aren’t entrepreneurs. They take no risk. They don’t even have to leave their suburban, no-winter environment. They don’t put up capital. They don’t risk sullying their reputations by investing their time in industries the future might despise; instead, they focus on boring consumer-web plays. They don’t go to foreign countries where they might not have all the creature comforts of the California suburbs. They don’t do the nuts-and-bolts operational grunt work that real entrepreneurs have to face (e.g. payroll, taxes) when they start new businesses, because their backers arrange it all for them. Even failure won’t disrupt their careers. If they fail, instead of making their $50-million payday sin this bubble cycle, they’ll have to settle for a piddling $750,000 personal take in an “acqui-hire”, a year in an upper-middle-management position, and an EIR gig. VC-backed “founders” take no real risk, but get rewarded immensely when things go their way. Heads, they win. Tails, they don’t lose.

Any time someone sets up a “heads I win, tails I-don’t-lose” arrangement, there’s a good bet that someone else is losing. Who? To some extent, it’s the passive capitalists whose funds are disbursed by VCs. Between careerist agents (VC partners) seeking social connection and status, and fresh-faced Disneypreneurs looking to justify their otherwise unreasonable career progress (due to their young age, questionable experience, and mediocrity of talent) what is left for the passive capitalist is a return inferior to that offered by a vanilla index fund. However, there’s another set of losers for whom I often prefer to speak, their plight being less well-understood: the engineers. Venture capitalists risk other peoples’ money. Founders risk losing access to the VCs if they do something really unethical. Engineers risk their careers. They’ve got more skin in the game, and yet their rewards are dismal.

If it’s such a raw deal to be a lowly engineer in a VC-funded startup (and it is) then why do so many people willingly take that offer? They might overestimate their upside potential, because they don’t know what questions to ask (such as, “If my 0.02% is really guaranteed to be worth $1 million in two years, then why do venture capitalists value the whole business at only $40 million?”). They might underestimate the passage of time and the need to establish a career before ageism starts hitting them. Most 22-year-olds don’t know what a huge loss it is not to get out of entry-level drudgery by 30. However, I think a big part of why it is so easy to swindle so many highly talented young people is the Disneyfication. The “cool” technology company, the Hooli, provides a halfway house for people just out of college. At Hooli, no one will make you show up for work at 9:00, or tell you not to wear sexist T-shirts, or expect you to interact decently with people too unlike you. You don’t even have to leave the suburbs of California. You won’t have to give up your car for Manhattan, your dryer for Budapest, your need to wear sandals in December for Chicago, or your drug habit for Singapore. It’s comfortable. There is no obvious social risk. Even the mean-spirited, psychotic policy of “stack ranking” is dressed-up as a successor to academic grading. (Differences glossed over are (a) that there’s no semblance of “meritocracy” in stack ranking; it’s pure politics, and a professor who graded as unfairly as the median corporate manager would be fired; (b) academic grading is mostly for the student’s benefit while stack-ranking scores are invariably to the worker’s detriment; and (c) universities genuinely try to support failing students while corporations use dishonest paperwork designed to limit lawsuit risk.) The comfort offered to the engineer by the Disney-fied tech world, which is actually more ruthlessly corporate (and far more undignified) than the worst of Wall Street, is completely superficial.

That doesn’t, of course, mean that it’s not real. Occasionally I’m asked whether I believe in God. Well, God exists. Supernatural beings may not, and the fictional characters featured in religious texts are almost certainly (if taken literally) pure nonsense, but the idea of God has had a huge effect on the world. It cannot be ignored. It’s real. The same of Silicon Valley’s style of “entrepreneurship”. Silicon Valley breathes and grows because, every year, an upper class of founders and proto-founders are given a safe, painless path to “entrepreneurial glory” and a much larger working class of delusional engineers are convinced to follow them. It really looks like entrepreneurship.

I should say one thing off the bat: Disneypreneurs are not the same thing as wantrapreneurs. You see more of the second type, especially on the East Coast, and it’s easy to conflate the two, but the socioeconomic distance is vast. The wantrapreneur can talk a big game, but lacks the drive, vision, and focus to ever amount to anything. He’s the sort of person who’s too arrogant to work for someone else, but can’t come up with a convincing reason why anyone should work for him, and doesn’t have the socioeconomic advantages that’d enable him to get away with bullshit. Except in the most egregious bubble times, he wouldn’t successfully raise venture capital, not because VCs are discerning but because the wantrapreneur usually lacks sufficient vision to learn how to do even that. Quite sadly, wantrapreneurs sometimes do find acolytes among the desperate and the clueless. They “network” a lot, sometimes find friends or relatives clueless enough to bankroll them, and produce little. Almost everyone has met at least one. There’s no barrier to entry in becoming a wantrapreneur.

Like a wantrapreneur, Disneypreneurs lack drive, talent, and willingness to sacrifice. The difference is that they still win. All the fucking time. Even when they lose, they win. Evan Spiegel (Snapchat) and Lucas Duplan (Clinkle) are just two examples, but Sean Parker is probably the most impressive. If you peek behind the curtain, he’s never actually succeeded at anything, but he’s a billionaire. They float from one manufactured success to another, build impressive reputations despite adding very little value to anything. They’re given the resources to take big risks and, when they fail, their backers make sure they fail up. Being dropped into a $250,000/year VP role at a more successful portfolio company? That’s the worst-case outcome. Losers get executive positions and EIR gigs, break-evens get acqui-hired into upper-six-figure roles, and winners get made.

One might ask: how does one become a Disneypreneur? I think the answer is clear: if you’re asking, you probably can’t. If you’re under 18, your best bet is to get into Stanford and hope your parents have the cardiac fortitude to see the tuition bill and not keel over. If you’re older, you might try out the (admirably straightforward, and more open to middle-class outsiders than traditional VC) Y Combinator. However, I think that it’s obvious that most people are never going to have the option of Disneypreneurship, and there’s a clear reason for that. Disneypreneurship exists to launder money (and connections, and prestige, and power; but those are highly correlated and usually mutually transferrable) for the upper classes, frank parasitism from inherited wealth being still socially unacceptable. The children of the elites must seem to work under the same rules as everyone else. The undeserving, mean-reverting progeny of the elite must be made to appear like they’ve earned the status and wealth their parents will bequeath upon them.

Elite schools were once intended toward this end. They were a prestige (multiple meanings intended) that appeared, from the outside, to be a meritocracy. However, this capacity was demolished by an often-disparaged instrument, the S.A.T. Sometimes, I’ll hear a knee-jerk leftist complain about the exam’s role in educational inequality, citing (correctly) the ability of professional tutoring (“test prep”, a socially useless service) to improve scores. In reality, the S.A.T. isn’t creating or increasing socioeconomic injustices in terms of access to education; it merely measures some of them. The S.A.T. was invented with liberal intentions, and (in fact) succeeded. After its inception in the 1920s, “too many” Jews were admitted to Ivy League colleges, and much of the “extracurricular” nonsense involved in U.S. college admissions was invented in a reaction to that. Over the following ninety years, there’s been a not-quite-monotonic movement toward meritocracy in college admissions. If I had to guess, college admissions are a lot more meritocratic than 90 years ago (and, if I’m wrong, it’s not because the admissions process is classist but because it’s so noise-ridden, thanks to technology enabling the application of a student to 15-30 colleges; 15 years ago, five applications was considered high). The ability-to-pay factor, however, keeps this meritocracy from being realized. Ties are, observably, broken on merit and there is enough meritocracy in the process to threaten the existing elite. The age in which a shared country-club membership of parent and admissions officer ensured a favorable decision is over. Now that assurance requires a building, which even the elite cannot always afford.

These changes, and the internationalization of the college process, and those pesky leftists who insist on meritocracy and diversity, have left the ruling classes unwilling to trust elite colleges to launder their money. They’ve shifted their focus to the first few years after college: first jobs. However, most of these well-connected parasites don’t know how to work and certainly can’t bear the thought of their children suffering the indignity of actually having to earn anything, so they have to bump their progeny automatically to unaccountable upper-management ranks. The problem is that very few people are going to respect a talentless 22-year-old who pulls family connections to get what he wants, and who gets his own company out of some family-level favor. Only a California software engineer would be clueless enough to follow someone like that– if that person calls himself “a founder”.

Why programmers can’t make any money: dimensionality and the Eternal Haskell Tax

To start this discussion, I’m going to pull down a rather dismal tweet from Chris Allen (@bitemyapp):

For those who don’t know, Haskell is a highly productive, powerful language that enables programmers (at least, the talented ones) to write correct code quickly: at 2 to 5 times the development speed of Java, with similar performance characteristics, and fewer bugs. Chris is also right on the observation that, in general, Haskell jobs don’t pay as well. If you insist on doing functional programming, you’ll make less money than people who sling C++ at banks with 30-year-old codebases. This is perverse. Why would programmers be economically penalized for using more powerful tools? Programmers are unusual, compared to rent-seeking executives, in actually wanting to do their best work. Why is this impulse penalized?

One might call this penalty “the Haskell Tax” and, for now, that’s what I’ll call it. I don’t think it exists because companies that use Haskell are necessarily cheaper or greedier than others. That’s not the case. I think the issue is endemic in the industry. Junior programmer salaries are quite high in times like 2014, but the increases for mid-level and senior programmers fall short of matching their increased value to the business, or even the costs (e.g. housing, education) associated with getting older. The only way a programmer can make money is to develop enough of a national reputation that he or she can create a bidding war. That’s harder to do for one who is strongly invested in a particular language. It’s not Haskell’s fault. There’s almost certainly a Clojure Tax and an Erlang Tax and a Scala Tax.

Beyond languages, this applies to any career-positive factor of a job. Most software jobs are career-killing, talent-wasting graveyards and employers know this, so when there’s a position that involves something interesting like machine learning, green-field projects, and the latest tools, they pay less. This might elicit a “well, duh” response, insofar as it shouldn’t be surprising that unpleasant jobs pay well. The reason this is such a disaster is because of its long-term effect, both on programmers’ careers and on the industry. Market signals are supposed to steer people toward profitable investment, but in software, it seems to fall the other way. Work that helps a programmer’s career is usually underpaid and, under the typical awfulness of closed allocation, jealously guarded, politically allocated, and usually won through unreasonable sacrifice.

Why is the Haskell Tax so damning?

As I said, the Haskell Tax doesn’t apply only to Haskell. It applies to almost all software work that isn’t fish-frying. It demolishes upper-tier salaries. One doesn’t, after all, get to be an expert in one’s field by drifting. It takes focus, determination, and hard work. It requires specialization, almost invariably. With five years of solid experience, a person can add 3 to 50 times more value than the entry-level grunt. Is she paid for that? Almost never. Her need to defend her specialty (and refuse work that is too far away from it) weakens her position. If she wants to continue in her field, there are a very small number of available jobs, so she won’t have leverage, and she won’t make any money. On the other hand, if she changes specialty, she’ll lose a great deal of her seniority and leverage, she’ll be competing with junior grunts, and so she won’t make any money either. It’s a Catch-22.

This puts an economic weight behind the brutality and incivility of closed allocation. It deprives businesses of a great deal of value that their employees would otherwise freely add. However, it also makes people less mobile, because they can’t move on to another job unless a pre-defined role exists matching their specialties. In the long run, the effect of this is to provide an incentive against expertise, to cause the skills of talented programmers to rot, and to bring the industry as a whole into mediocrity.

Code for the classes and live with the masses. Code for the masses and live… with the masses.

Artists and writers have a saying: sell to the masses, and live with the classes; sell to the classes, and live with the masses. That’s not really a statement about social class as about the low economic returns of high-end work. Poets don’t make as much money as people writing trashy romance novels. We might see the Haskell Tax as an extension of this principle. Programmers who insist on doing only the high-end work (“coding for the classes”) are likely to find themselves either often out of work, or selling themselves at a discount.

Does this mean that every programmer should just learn what is learned in 2 years at a typical Java job, and be done with it? Is that the economically optimal path? The “sell to the masses” strategy is to do boring, line-of-business, grunt work. Programmers who take that tack still live with the masses. That kind of programming (parochial business logic) doesn’t scale. There’s as much work, for the author, in writing a novel for 10 people as 10 million; but programmers don’t have that kind of scalability, and the projects where there are opportunities for scaling, growth, and multiplier-type contributions are the “for the classes” projects that every programmer wants to do (we already discussed why those don’t pay). So, programming for the masses is just as much of a dead end, unless they can scale up politically– that is, become a manager. At that point, they can sell code, but they don’t get to create it. They become ex-technical, and ex-technical management (with strongly held opinions, once right but now out of date) can be just as suboptimal as non-technical management.

In other words, the “masses” versus “classes” problem looks like this, for the programmer: one can do high-end work and be at the mercy of employers because there’s so little of it to go around, or to low-end commodity work that doesn’t really scale. Neither path is going to enable her to buy a house in San Francisco.

Dimensionality

One of the exciting things about being a programmer is that the job always changes. New technologies emerge, and programmers are expected to keep abreast of them even when their employers (operating under risk aversion and anti-intellectualism) won’t budget the time. What does it mean to be a good programmer? Thirty years ago, it was enough to know C and how to structure a program logically. Five years ago, a software engineer was expected to know a bit about a Linux, MySQL, a few languages (Python, Java, C++, Shell) and the tradeoffs among them. In 2014, the definition of “full stack” has grown to the point that almost no one can know all of it. Andy Shora (author of the afore-linked essay) puts it beautifully, on the obnoxiousness of the macho know-it-all programmer:

I feel the problem for companies desperate to hire these guys and girls, is that the real multi-skilled developers are often lost in a sea of douchebags, claiming they know it all.

Thirty years ago, there was a reasonable approximation of a linear ordering on programmer skill. If you could write a C compiler, understood numerical stability, and could figure out how to program in a new language or for a new platform by reading the manuals, you were a great programmer. If you needed some assistance and often wrote inefficient algorithms, you were either a junior or mediocre. In 2014, it’s not like that at all; there’s just too much to learn and know! I don’t know the first thing, for example, about how to build a visually appealing casual game. I don’t expect that I’d struggle as much with graphics as many do, because I’m comfortable with linear algebra, and I would probably kill it when it comes to AI and game logic, but the final polish– the difference between Candy Crush and an equivalent but less “tasty” game– would require someone with years of UI/UX experience.

The question of, “What is a good programmer?”, has lost any sense of linear ordering. The field is just too vast. It’s now an N-dimensional space. This is one of the things that makes programming especially hostile to newcomers, to women, and non-bullshitters of all stripes. The question of which of those dimensions matter and which don’t is political, subjective, and under constant change. One year, you’re a loser if you don’t know a scripting language. The next, you’re a total fuckup if you can’t explain what’s going on inside the JVM. The standards change at every company and frequently, leaving most people not only at a loss regarding whether they are good programmers, but completely without guidance about how to get there. This also explains the horrific politics for which software engineering is (or, at least, ought to be) notorious. Most of the “work” in a software company is effort spent trying to change the in-house definition of a good programmer (and, to that end, fighting incessantly over tool choices).

I don’t think that dimensionality is a bad thing. On the contrary, it’s a testament to the maturity and diversity of the field. The problem is that we’ve let anti-intellectual, non-technical businessmen walk in and take ownership of our industry. They demand a linear ordering of competence (mostly, for their own exploitative purposes). It’s the interaction between crass commercialism and dimensionality that causes so much pain.

Related to this is the Fundamental Hypocrisy of Employers, a factor that makes it damn hard for a programmer to navigate this career landscape. Technology employers demand specialization in hiring. If you don’t have a well-defined specialty and unbroken career progress toward expertise in that field, they don’t want to talk to you. At the same time, they refuse to respect specialties once they’ve hired people, and people who insist on protecting their specialties (which they had to do to get where they are) are downgraded as “not a team player”. Ten years of machine learning experience? Doesn’t matter, we need you to fix this legacy Rails codebase. It’s ridiculous, but most companies demand an astronomically higher quality of work experience than they give out. The result of this is that the game is won by political favorites and self-selling douchebags, and most people in either of those categories can’t really code.

The Eternal Haskell Tax

The Haskell Tax really isn’t about Haskell. Any programmer who wishes to defend a specialty has a smaller pool of possible jobs and will generally squeeze less money out of the industry. As programming becomes more specialized and dimensional, the Haskell Tax problem affects more people. The Business is now defining silos like “DevOps” and “data science” which, although those movements began with good intentions, effectively represent the intentions of our anti-intellectual colonizers to divide us against each other into separate camps. The idea (which is fully correct, by the way) that a good Haskell programmer can also be a good data scientist or operations engineer is threatening to them. They don’t want a fluid labor market. Our enemies in The Business dislike specialization when we protect our specialties (they want to make us interchangeable, “full stack” generalists) but, nonetheless, want to keep intact the confusion and siloization that dimensionality creates. If the assholes in charge can artificially disqualify 90 percent of senior programmers from 90 percent of senior programming jobs based on superficial differences in technologies, it means they can control us– especially if they control the assignment of projects, under the pogrom that is closed allocation– and (more importantly) pay us less.

The result of this is that we live under an Eternal Haskell Tax. When the market favors it, junior engineers can be well-paid. But the artificial scarcities of closed allocation and employer hypocrisy force us into unreasonable specialization and division, making it difficult for senior engineers to advance. Engineers who add 10 times as much business value as their juniors are lucky to earn 25 percent more; they, as The Business argues, should consider themselves fortunate in that they “were given” real projects!

If we want to fix this, we need to step up and manage our own affairs. We need to call “bullshit” on the hypocrisy of The Business, which demands specialization in hiring but refuses to respect it internally. We need to inflict hard-core Nordic Indignation on closed allocation and, in general, artificial scarcity. Dimensionality and specialization are not bad things at all (on the contrary, they’re great) but we need to make sure that they’re properly managed. We can’t trust this to the anti-intellectual colonial authorities who currently run the software industry, who’ve played against us at every opportunity. We have to do it ourselves.

Why there are so few AI jobs

Something began in the 1970s that has been described as “the AI winter”, but to call it that is to miss the point, because the social illness it represents involves much more than artificial intelligence (AI). AI research was one of many casualties that came about as anti-intellectualism revived itself and society fell into a diseased state.

One might call the “AI winter” (which is still going on) an “interesting work winter” and it pertains to much more of technology than AI alone, because it represented a sea change in what it meant to be a programmer. Before the disaster, technology jobs had an R&D flavor, like academia but with better pay and less of the vicious politics. After the calamitous 1980s and the replacement of R&D by M&A, work in interesting fields (e.g. machine learning, information retrieval, language design) became scarce and over 90% of software development became mindless, line-of-business makework. At some point, technologists stopped being autonomous researchers and started being business subordinates and everything went to hell. What little interesting work remained was only available in geographic “super-hubs” (such as Silicon Valley) where housing prices are astronomical compared to the rest of the country. Due to the emasculation of technology research in the U.S., economic growth slowed to a crawl, and the focus of the nation’s brightest minds turned to creation of asset bubbles (seen in 1999, 2007, and 2014) rather than generating long-lasting value.

Why did this happen? Why did the entrenched public- and private-sector bureaucrats (with, even among them, the locus of power increasingly shifting to private-sector bureaucrats, who can’t be voted out of office) who run the world lose faith in the research being done by people much smarter, and who work much harder, than them? The answer is simple. It’s not even controversial. End of the Cold War? Nah, it began before that. At fault is the lowly perceptron.

Interlude: a geometric puzzle

This is a simple geometry puzzle. Below are four points at the corners of the square, colored (and numbered) like so:

0 1
1 0

Is it possible to draw a line that separates the red points (0′s) from the green points (1′s)?

The answer is that it’s not possible. Any separating line would have to separate two points from each other. Now draw a circle passing through all four points. Any line can intersect that circle at no more than two points. Therefore, a line separating two points from the other two would have to separate two adjacent points, which would be of opposing colors. It’s not possible. Another way to say this is that the classes (colors) aren’t linearly separable.

What is a perceptron?

“Perceptron” is a fancy name given to a mathematical function with a simple description. Let w be a known “weight” vector (if that’s an unfamiliar term, a list of numbers) and x be an input “data” vector of the same size, with the caveat that x[0] = 1 (a “bias” term) always. The perceptron, given w, is a virtual “machine” that computes, for any given input x, the following:

  • 1, if w[0]*x[0] + … + w[n]*x[n] > 0,
  • 0, if w[0]*x[0] + … + w[n]*x[n] < 0.

In machine learning terms, it’s a linear classifier. If there’s a linear function that cleanly separates the “Yes” class (the 1 values) from the “No” class (the 0 values) it can be expressed as a perceptron. There’s an elegant algorithm for, in that linearly separable case, finding a working weight vector. It always converges.

A mathematician might say, “What’s so interesting about that? It’s just a dot product being passed through a step function.” That’s true. Perceptrons are very simple. A single perceptron can solve more decision problems than one might initially think, but it can’t solve all of them. It’s too simple a model.

Limitations

Let’s say that you want to model an XOR (“exclusive or”) gate, corresponding to the following function:

| in_1 | in_2 | out |
+------+------+-----+
|   0  |   0  |  0  |
|   0  |   1  |  1  |
|   1  |   0  |  1  |
|   1  |   1  |  0  |
+------+------+-----+

One might recognize that this is identical to the “brainteaser” above, with in_1 and in_2 corresponding to the x– and y– dimensions in the coordinate plane. This is the same problem. This function is nonlinear; it could be expressed as f(x, y) = x + y – 2xy. and that’s arguably the simplest representation of it that works. A separating “plane” in the 2-dimensional space of the inputs would be a line, and there’s no line separating the two classes. It’s mathematically obvious that the perceptron can’t do it. I showed this, above, using high-school geometry.

To a mathematician, this isn’t surprising. Marvin Minsky pointed out the mathematically evident limitations of a single perceptron. One can model intricate mathematical functions with more complex networks of perceptrons and perceptron-like units, called artificial neural networks. They work well. One can also, using what are called “basis expansions”, generate further dimensions from existing data in order to create a higher-dimensional space in which linear classifiers still work. (That’s what people usually do with support vector machines, which provide the machinery to do so efficiently.) For example, adding xy as a third “derived” input dimension would make the classes (0′s and 1′s) linearly separable. There’s nothing mathematically wrong with doing that; it’s something that statisticians do when they want to build complex models but still have some of the analytic properties of simpler ones, like linear regression or nearest-neighbor modeling.

The limitations of the single perceptron do not invalidate AI. At least, they don’t if you’re a smart person. Everyone in the AI community could see the geometrically obvious limitation of a single perceptron, and not one of them believed that it came close to invalidating their work. It only proved that more complex models were needed for some problems, which surprised no one. Single-perceptron models might still be useful for computational efficiency (in the 1960s, computational power was about a billion times as expensive as now) or because the data don’t support a more complex model; they just couldn’t learn or model every pattern.

In the AI community, there was no scandal or surprise. That some problems aren’t linearly separable is not surprising. However, some nerd-hating non-scientists (especially in business upper management) took this finding to represent more than it actually did.

They fooled us! A brain with one neuron can’t have general intelligence!

The problem is that the world is not run, and most of the wealth in it is not controlled, by intelligent people. It’s run by social-climbing empty-suits who are itching for a fight and would love to take some “eggheads” down a notch. Insofar as an artificial neural network models a brain, a perceptron models a single neuron, which can’t be expected to “think” at all. Yet the fully admitted limitations of a single perceptron were taken, by the mouth-breathing muscleheads who run the world, as an excuse to shit on technology and pull research funding because “AI didn’t deliver”. That produced an academic job market that can only be described as a pogrom, but it didn’t stop there. Private-sector funding dried up as short-term, short-tempered management came into vogue.

To make it clear, no one ever said that a single perceptron can solve every decision problem. It’s a linear model. That means it’s restricted, intentionally, to a small subspace of possible models. Why would people work with a restricted model? Traditionally, it was for a lack of data. (We’re in the 1960s and ’70s, when data was contained on physical punch cards and a megabyte weighed something and a disk drive cost more than a car.) If you don’t have a lot of data, you can’t build complex models. For many decision problems, the humble perceptron (like its cousins, logistic regression and support vector machines) did well and, unlike other computationally intensive linear classification methods (such as logistic regression, which requires gradient descent, or a variant thereof, over the log-likelihood surface; or such as the support vector machine, which are a quadratic programming problem that we didn’t know how to solve efficiently until the 1990s) it could be trained with minimal computational expense, in a bounded amount of time. Even today, linear models are surprisingly effective for a large number of problems. For example, the first spam classifiers (Naive Bayes) operated using a linear model, and it worked well. No one was claiming that a single perceptron was the pinnacle of AI. It was something that we could build cheaply on 1970-era hardware and that could build a working model on many important datasets.

Winter war

Personally, I don’t think that the AI Winter was an impersonal, passive event like the changes of seasons. Rather, I think it was part of a deliberate resurgence of anti-intellectualism in a major cultural war– one which the smart people lost. The admitted limitations of one approach to automated decision-making gave the former high school bullies, now corporate fat cats, all the ammo they needed in order to argue that those “eggheads” weren’t as smart as they thought they were. None of them knew exactly what a perceptron or an “XOR gate” were, but the limitation that I’ve described was morphed into “neural networks can’t solve general mathematical problems” (arguably untrue) and that turned into “AI will never deliver”. In the mean-spirited and anti-liberal political climate of the 1980s, this was all that anyone needed as an excuse to cut public funding. The private sector not only followed suit, but amplified the trend. The public cuts were a mix of reasonable fiscal conservatism and mean-spirited anti-research sentiment, but the business elites responded strongly to (and took to a whole new level) the mean-spirited aspect, flexing their muscles as elitism (thought vanquished in the 1930s to ’50s) became “sexy” again in the Reagan Era. Basic research, which gave far too much autonomy and power to “eggheads”, was slashed, marginalized, and denigrated.

The claim that “AI didn’t deliver” was never true. What actually happened is that we solved a number of problems, once thought to require human intelligence, with a variety of advanced statistical means as well as some insights from fields like physics, linguistics, ecology and economics. Solving problems demystified them. Automated mail sorting, once called “artificial intelligence”, became optical character recognition. This, perhaps, was part of the problem. Successes in “AI” were quickly put into a new discipline. Even modern practitioners of statistical methods are quick to say that they do machine learning, not AI. What was actually happening is that, while we were solving specific computational problems once thought to require “intelligence”, we found that our highly specialized solutions did well on the problems they were designed for, and could be adapted to similar problems, but with very slow progress toward general intelligence. As it were, we’ve learned in recent decades that our brains are even more complicated than we thought, with a multitude of specialized modules. That no specific statistical algorithm can replicate all of them, working together in real time, shouldn’t surprise anyone. Is this an issue? Does it invalidate “AI” research? No, because most of those victories, while they fell short of replicating a human brain, still delivered immense economic value. Google, although it eventually succumbed to the sociological fragility and failure that inexorably follow closed allocation, began as an AI company. It’s now worth over $360 billion.

Also mixed in with the anti-AI sentiment is the religious aspect. It’s still an open and subjective question what human intelligence really is. The idea that human cognition could be replicated by a computer offended religious sentiments, even though few would consider automated mail sorting to bear on unanswerable questions about the soul. I’m not going to go deep into this philosophical rabbit hole, because I think it’s a waste of time to debate why people believe AI research (or, for a more popular example, evolution by natural selection) to offend their religious beliefs. We don’t know what qualia is or where it comes from. I’ll just leave it at this. If we can use advanced computational techniques to solve problems that were expensive, painful, or impossible given the limitations of human cognition, we should absolutely do it. Those who object to AI on religious grounds fear that advanced computational research will demystify cognition and bring about the end of religion. Ignoring the question of whether an “end of religion” is a bad thing, or what “religion” is, there are two problems with this. First, if there is something to us that is non-material, we won’t be able to replicate it mechanically and there is no harm, to the sacred, in any of this work. Second, computational victories in “AI” tend to demystify themselves and the subfield is no longer considered “AI”. Instead, it’s “optical character recognition” or “computer game-playing”. Most of what we use on a daily basis (often behind the scenes, such as in databases) comes from research that was originally considered “artificial intelligence”.

Artificial intelligence research has never told us, and will never tell us, whether it is more reasonable to believe in gods and religion or not to believe. Religion is often used by corrupt, anti-intellectual, politicians and clerics to rouse sentiment against scientific progress, as if automation of human grunt work were a modern-day Tower of Babel. Yet, to show what I mean by AI victories demystifying themselves, almost none would hesitate to use Google, a web-search service powered by AI-inspired algorithms.

Why do the anti-intellectuals in politics and business wish to scare the public with threats of AI-fueled irreligion and secularism (as if those were bad things)? Most of them are intelligent enough to realize that they’re making junk arguments. The answer, I think, is about raw political dominance. As they see it, the “nerds” with their “cushy” research jobs can’t be allowed to (gasp!) have good working conditions.

The sad news is that the anti-intellectuals are likely to take the economy and society down with them. In the 1960s, when we were putting billions of dollars into “wasteful” research spending, the economy grew at a record pace. The world economy was growing at 5.7 percent per year, and the U.S. economy was the envy of the world. Now, in our spartan time of anti-intellectualism, anti-science sentiment, and corporate elitism, the economy is sluggish and the society is stagnant– all because the people in charge can’t stand to see “eggheads” win.

Has AI “delivered”?

If you’re looking to rouse religious fear and fury, you might make a certain species of fantastic argument against “artificial intelligence”. The truth of the matter, however, is that while we’ve seen domain-specific superiority of machines over human intelligence in rote processes, we’re still far from creating an artificial general intelligence, i.e. a computational entity that can exhibit the general learning capability of a human. We might never do it. We might not need to and, I would argue, we should not if it is not useful.

In a way, “artificial intelligence” is a defined-by-exclusion category of “computational problems we haven’t solved yet”. Once we figure out how to make computers better at something than humans are, it becomes “just computation” and is taken for granted. Few believe they’re using “an AI” when they use Google for web search, because we’re now able to conceive of the computational work it does as mechanical rather than “intelligent”.

If you’re a business guy just looking to bully some nerds, however, you aren’t going to appeal to religion. You’re going to make the claim that all this work on “artificial intelligence” hasn’t “delivered”. (Side note: if someone uses “deliver” intransitively, as business bullies are wont to do, you should punch that person in the face.) Saying someone or something isn’t “delivering” is a way to put false objectivity behind a claim that means nothing other than “I don’t like that person”. As for AI, it’s true that artificial general intelligence has eluded us thus far, and continues to do so. It’s an extremely hard problem: far harder than the optimists among us thought it would be, fifty years ago. However, the CS research community has generated a hell of a lot of value along the way.

The disenchantment might be similar to the question about “flying cars”. We actually have them. They’re called small airplanes. In the developed world, a person of average means can learn how to fly one. They’re not even that much more expensive than cars. The reason so few people use airplanes for commuting is that it just doesn’t make economic sense for them: the savings of time don’t justify increased fuel and maintenance costs. But a middle-class American or European can, if she wants, have a “flying car” right now. It’s there. It’s just not as cheap or easy to use as we’d like. With artificial intelligence, that research has brought forth a ridiculous number of victories and massive economic growth. It just hasn’t brought forth an artificial general intelligence. That’s fine; it’s not clear that we need to build one in order to get the immense progress that technologists create when given the autonomy and support.

Back to the perceptron

One hard truth I’ve learned is that any industrial effort will have builders and politicians. It’s very rare that someone is good at both. In the business world, those unelected private-sector politicians are called “executives”. They tend, for a variety of reasons, to put themselves into pissing contests with the builders (“eggheads”) who are actually making stuff. One time-tested way to show up the builders is to take something that is obviously true (leading the builders to agree with the presentation) but present it out of context in a way that is misleading.

The incapacity of the single perceptron at general mathematical modeling is a prime example of this. Not one AI researcher was surprised that such a simple model couldn’t describe all patterns or equational relationships. The fact that can be proven (as I did) with high school geometry. That a single perceptron can’t model a key logical operation is, as above, obviously true. The builders knew it, and agree. Unfortunately, what the builders failed to see was that the anti-intellectual politicians were taking this fact way out of context, using the known limitations of a computational building block to ascribe limitations (that did not exist) to general structures. This led to the general dismantling of public, academic, and private support for technological research, an anti-intellectual and mean-spirited campaign that continues to this day.

That’s why there are so few AI jobs.