This is directly in response to Matt Aimonetti’s “Engineers Suck at Finding the Right Jobs“, because I disagree that the problem resides solely with engineers. Rather, I think the problem is bilateral. An equally strong argument could be made that there’s an immense wealth of engineering talent (or, at least, potential) out there, but that our contemporary business leadership lacks the vision, creativity, and intelligence to do anything with it.
Don’t get me wrong: I basically agree with what he is saying. Most software engineers do a poor job at career management. A major part of the problem is that the old-style implicit contract between employers and employees has fallen to pieces, and people who try to follow the old rules will shortchange themselves and fail in their careers. In the old world, the best thing for a young person to do was to take a job– any job– at a reputable company and just be seen around the place, and eventually graduate into higher quality of work. Three to five years of dues-paying grunt work (that had little intrinsic career-building value, but fulfilled a certain social expectation) was the norm, but this cost only had to be paid once. The modern world is utterly different. Doing grunt work does nothing for your career. There are people who get great projects and advance quickly, and others who get bad projects and never get out of the mud. Our parents lived in a world where “90 percent is showing up”, whereas we live in one where frequent job changes are not only essential to a good career, but often involuntary.
Software engineering is full of “unknown unknowns”, which means that most of us have a very meager understanding of what we don’t know, and what our shortcomings are. We often don’t know what we’re missing. It’s also an arena in which the discrepancy between the excellent and the mediocre isn’t a 20 to 40 percent increase, but a factor of 2 to 100. Yet to become excellent, an engineer needs excellent work, and there isn’t much of that to go around, because most managers and executives have no vision. In fact, there’s so little excellent work in software that what little there is tends to be allocated as a political favor, not given to those with the most talent. The most important skill for a software engineer to learn, in the real world, is how to get allocated to the best projects. In fact, what I would say distinguishes the most successful engineers is that they develop, at an early age, the social skills to say “no” to career-negative grunt work without firing oneself in the process. That’s how they take the slack out of their careers and advance quickly.
That’s not the same as picking “the right jobs”, because engineers don’t actually apply to specific work sets when they seek employment, but to companies and managers. Bait-and-switch hiring practices are fiendishly common, and many companies are all-too-willing to allocate undesirable and even pointless work to people in the “captivity interval”, which tends to span from the 3rd to the 18th month of employment, at which point leaving will involve a damaging short job tenure on the resume. (At less than 3 months, the person has the option of omitting that job, medium-sized gaps being less damaging than short-term jobs, which suggest poor performance.) I actually don’t think there’s any evidence to indicate that software engineers do poorly at selecting companies. Where I think they are abysmal is at the skill of placing themselves on high-quality work once they get into these companies.
All of this said, this matter raises an interesting question: why is there so much low-quality work in software? I know this business well enough to know that there aren’t strong business reasons for it to be that way. High-quality work is, although more variable, much more profitable in general. Companies are shortchanging themselves as much as their engineers by having a low-quality workload. So why is there so little good work to go around?
I’ve come to the conclusion that most company’s workloads can be divided into four quadrants based on two variables. The first is whether the work is interesting or unpleasant. Obviously, “interestingness” is subjective, so I tend to assume that work should be called interesting if there is someone out there who would happily do it for no more (and possibly less) than a typical market salary. Some people don’t have the patience for robotics, but others love that kind of work, so I classify it in the “interesting” category, because I’m fairly confident that I could find someone who would love to do that kind of work. For many people, it’s “want-to” work. On the other hand, the general consensus is that there’s a lot of work that very few people would do, unless paid handsomely for it. That’s the unpleasant, “have-to” work.
The second variable is whether the work is essential or discretionary. Essential work involves a critical (and often existential) issue for the company. If it’s not done, and not done well, the company stands to lose a lot of money: millions to billions of dollars. Discretionary work, on the other hand, isn’t in the company’s critical path. It tends to be exploratory work, or support work that the firm could do without. For example, unproven research projects are discretionary, although they might become essential later on.
From these two variables, work can be divided into four quadrants:
Interesting and Essential (1st Quadrant): an example would be Search at Google. This work is highly coveted. It’s both relevant and rewarding, so it benefits an employee’s internal and external career goals. Sadly, there’s not a lot of this in most companies, and closed-allocation companies make it ridiculously hard to get it.
Unpleasant and Essential (2nd Quadrant): grungy tasks like maintenance of important legacy code. This is true “have-to” work: it’s not pleasant, but the company relies on it getting done. Boring or painful work generally doesn’t benefit an employee’s external career, so well-run companies compensate by putting bonuses and internal career benefits (visibility, credibility, promotions) on it: a market solution. These are “hero projects”.
Interesting and Discretionary (3rd Quadrant): often, this takes the form of self-directed research projects and is the domain of “20% time” and “hack days”. This tends to be useful for companies in the long term, but it’s not of immediate existential importance. Unless the project were so successful as to become essential, few people would get promoted based on their contributions in this quadrant. That said, a lot of this work has external career benefits, because it looks good to have done interesting stuff in the past, and engineers learn a lot by doing it.
Unpleasant and Discretionary (4th Quadrant): this work doesn’t look good in a promotion packet, and it’s unpleasant to perform. This is the slop work that most software engineers get because, in traditional managed companies, they don’t have the right to say “no” to their managers. The business value of this work is minimal and the total value (factoring in morale costs and legacy) is negative. 4th-Quadrant work is toxic sludge that should be avoided.
One of the reasons that I think open allocation is the only real option is that it eliminates the 4th-Quadrant work that otherwise dominates a corporate atmosphere. Under open allocation, engineers vote with their feet and tend to avoid the 4th-Quadrant death marches.
The downside of open allocation, from a managerial perspective, is that the non-coercive nature of such a regime means they have to incent people to work on 2nd-Quadrant work, often with promotions and large (six- or seven-figure) bonuses. It seems expensive. Closed allocation enables managers to get the “have-to” work done cheaply, but there’s a problem with that. Under closed allocation, people who are put on these unpleasant projects often get no real career compensation, because management doesn’t have to give them any. So the workers put on such projects feel put-upon and do a bad job of it. If the work is truly 2nd-Quadrant (i.e. essential) the company cannot afford to have it done poorly. It’s better to pay for it and get high quality than to coerce people into it and get garbage.
The other problem with closed allocation is that it eliminates the market mechanic (workers voting with their feet) that allows this quadrant structure to become visible at all, which means that management in closed-allocation companies won’t even know when it has a 4th-Quadrant project. The major reason why closed-allocation companies load up on the toxic 4th-Quadrant work is because they have no idea that it’s even there, nor how to get rid of it.
There’s no corporate benefit to 4th-Quadrant work. So what incentive is there to generate it? Middle management is to blame. Managers don’t care whether the work is essential or discretionary, because they just want the experience of “leading teams”. They’re willing to work on something less essential, where there’s less competition to lead the project (and also a higher chance of keeping one’s managerial role) because their careers benefit either way. They can still say they “led a team of 20 people”, regardless of what kind of work they actually oversaw. Middle managers tend to what little interesting stuff these discretionary projects have for themselves, placing themselves in the 3rd-Quadrant, while leaving the 4th-Quadrant work to their reports.
This is the essence of what’s wrong with corporate America. Closed allocation generates pointless work that (a) no one wants to do, and (b) provides no real business value to the company. It’s a bilateral lose-lose for the company and workers, existing only because it suits the needs of middlemen.
It’s common wisdom in software that 90 to 95 percent of software engineers are depressingly mediocre. I don’t know what the percentage is, but I find that to be fairly accurate, at least in concept. The bulk of software engineers are bad at their jobs. I disagree that this mediocrity is intrinsic. I think it’s a product of bad work environments, and the compounding effect of bad projects and discouragement over time. The reason there are so many incompetent software engineers out there is that the work they get is horrible. It’s not only that they lack the career-management skills to get better work; it’s also that good work isn’t available to them when they start out, and it becomes even less available over time as their skills decline and their motivation and energy levels head toward the crapper.
I don’t see any intrinsic reason why the world can’t have ten, or even a hundred, times as many competent software engineers as it has now, but the dumbed-down corporate environment that most engineers face will block that from coming to fruition.
There’s an incredible amount of potential engineering talent out there, and for the first time in human history, we have the technology to turn it into gold. Given this, why is so much of it being squandered?
