The Major Software Engineering Mistake That No One is Talking About and How You Can Take Advantage

Managers of software engineering teams are making one huge mistake. It prevails almost everywhere, from internal development teams in large corporations to smaller, independent software engineering firms whether foreign or domestic. If you understand it and learn how to avoid it, then it would give you a major advantage over your competitors. So, what is this mistake?

It is this.

Folks are under the misguided impression that software engineers are basically the same. A developer is a developer is a developer. The prevailing sentiment goes something like this.

If Developer A builds a button that executes some logic and displays a result, and Developer B also builds a button that executes the same logic and displays the same result, then Developer A and Developer B are equivalent. Therefore, it is better to hire the cheaper Developer.

Oh, how seductive and utterly destructive this line of thinking is. On the surface this assumption might seem to be quite reasonable but let me explain why it is a giant blunder.

Imagine, if you will, that you have a car in need of repair and you are able to make an exact copy of that car atom-for-atom, molecule-for-molecule, in the same way that you can make a copy of digital files.

Now imagine that you take both identical cars with the same defect to two different auto mechanics. Both mechanics then repair their respective vehicles so that the defect is completely removed. For each of the cars, you get in, turn on the ignition, drive it, and all seems fine.

Now here is the question.

Which of the mechanics did the better job or is their work indeed equivalent?

As a layman, how can you answer this question? If you don’t understand how to evaluate the job mechanically, then you are forced to assume that the only differentiator between the two mechanics is the price they charge for performing the repair, and so you would logically choose the cheaper mechanic.

But the truth is that not all auto mechanics are the same. Not all doctors are the same. Not all lawyers, teachers, taxi drivers, chefs, customer service representatives are the same. In terms of talent, skill, and prowess, some are on the high end of the bell curve, some are on the low end, but most are in the middle ‑ that is … mediocre.

Why then are software engineers treated like commodities?

The fact of the matter is that if you were to employ a third-party expert in auto mechanics, then the expert would be able to truly evaluate the quality of craftsmanship of the two cars in the repair scenario. You probably wouldn’t be surprised to be told that one of the mechanics did a better job than the other.

The harsh truth is that most software engineers produce low-to-medium quality code at best for a variety of reasons, mostly having nothing to do with their drive or work ethic. They love what they do and strive to do their very best. But despite their efforts, good intentions, and their improved code quality over time, they frequently produce code that causes the following symptoms of bad software architecture.

• Development takes a long time and is costly
• Changes cause unintended bugs
• Deadlines are rarely, if ever, met
• Give rise to “Tribal Knowledge” where only one or two key developers know everything
• New team members take a long time to ramp up and need current members to explain things
• Small changes require developers to code and deploy instead of users tweaking options/settings
• Some desired changes cannot be implemented because it requires “brain surgery” of the system
• Issues keep popping up requiring developers to intervene ‑ firefighting is the order of the day

These problems in your systems are a massive drain on the business. It is extremely expensive to be in this situation not only in terms of money spent but also in terms of opportunity cost. How much better off would your business be if you could have launched that initiative two quarters ago? Your business could have enjoyed two quarters where money wasn’t burned and instead money was being made.

If you understand and accept the fact that not all developers are the same and that left to their own devices most of them will unintentionally build bad software, then you can take steps to avoid it. But this begs the question.

If you don’t know how to tell the difference between a good developer and a bad developer, then how can you avoid making this mistake?

Let’s unpack that.

Building quality software is in part a delicate balance of trust and fulfillment between the stakeholders and engineers. But to address the problem of developers writing bad code there must be an appreciation of one very vital role in software engineering and that is the role of the Software Architect.

So, what exactly is a Software Architect?

To put it simply, software is basically a black box that accepts some inputs and emits the expected outputs. All the mechanics underlying the functionality inside that box is a function of software engineering and its quality is dictated by the quality of its internal architecture. A Software Architect, therefore, is a high-caliber software engineer who leads, guides, and ensures the quality of the code inside that box.

The Software Architect role is under-appreciated because Solution Architects get all the attention and are treated like celebrity rock stars. Why is that? To understand why this is the case, it helps to know the difference between a Software Architect and a Solution Architect.

Whereas a Software Architect deals with the architecture inside of the box, a Solution Architect deals with how boxes talk to boxes.

In other words, systems at scale require a design where the system is made up of many pieces of software that work together. Done right, these systems allow for large scale implementations that can handle many thousands or millions of users. Think of the Googles, Twitters, and Amazons of the world. Dealing with massive concurrent users is a challenge on another level. But these systems are the exception. Even the largest corporations don’t have massive concurrent users for all their systems. Companies like U.S. Foods, United Airlines, Walgreens, McDonald’s, all have some publicly facing systems that require scale for sure, but they all have many, many smaller internal systems to facilitate the various operations that don’t see the public light of day. These smaller systems serve internal departments or teams of comparatively small groups of people. Sometimes as small as a few people, sometimes a hundred people, but not thousands or millions.

In general, these smaller systems are developed by people who are not good Software Architects, or they are on teams that don’t have the role of a Software Architect. They have Lead Developers and Solution Architects, but the role of Software Architect that Developers and Lead Developers report to for architectural matters is frequently non-existent. Consequently, they basically have rookies making critical software architectural decisions because of the misguided notion that a developer who can code and get a button or screen working is basically the same thing.

To make matters worse, the good Software Architects aspire to and then frequently achieve the gaining of roles as Solution Architects. This is logical because otherwise they are treated like any other developer despite their elite architectural prowess. As a Solution Architect they are treated like gold and paid accordingly.

Any business that appreciates the Software Architect role as much as a Solution Architect role will gain a comparative advantage over businesses that do not. If you want to ensure that your software development teams do not build bad code, find the software equivalent of the expert auto mechanic, which is a bona fide Software Architect and do the following.

1. Empower the Software Architect with decision-making authority on software architecture
2. Hold the Software Architect accountable not only for the technical quality of code but also for the chemistry of the team
3. Pay the Software Architect well ‑ do not go cheap on this role
4. Ensure the Software Architect has the time and resources to work toward a better future. Don’t use this person as another developer who can assist in day-to-day development tasks
5. Account for the cost of building common libraries that the Software Architect will need to build
6. Ensure the Software Architect is also a mentor who freely and enthusiastically spreads the knowledge and experience of good software development practices
7. If there are multiple teams then ensure each team has a Lead Developer who is the liaison between the team and the Software Architect
8. With a group of Lead Developers, then each Lead Developer should be a developer for the Software Architect whose role also entails extracting the best answers from Lead Developers as well as her/him self
9. Ensure that the role of the Lead Developers is to
   • Work with the Software Architect to formulate architectural decisions
   • Lead their respective teams on the implementation details
   • Participate in the development of shared libraries with the Software Architect and other Lead Developers

Once you do this, then the team(s) naturally form(s) a farm system of engineers that eventually become skilled architects. If your Software Architect leaves or retires, then this culture of good code ensures that there will be a line of succession with one or more worthy Leads to become the new Software Architect.

This is not possible without the Software Architect role and is certainly not possible when the organizational structure of engineers is flattened mixing low-skill, medium-skill, and expert engineers at the same level all with the “Senior Developer” title.

That leaves one last problem to solve.

How do you recognize an elite Software Architect when you see one?

Well, that is the subject of another article for another day. In the meantime, good luck.