Principles of Software Pathology
A practical guide to improving Software Consultancy
Abstract: Compared to medical treatment, software consulting remains a rudimentary activity lacking standardized practices with predictable outcomes. I will present introductory concepts of Human Pathology to show how software developers and consultants can learn from them. Additionally, I will propose a methodology for applying these concepts to evaluate, diagnose and improve software projects. Finally, I’ll share some reflections of the actions needed to be taken by the software guild to adopt the scientific method in our daily practice.
1. Introduction
Pathology is the science that studies human suffering, it’s an elementary subject in the medical training to diagnose and treat patients. Their modern foundations can be traced to the invention of the microscope and the consequent discovery that microorganisms are the main cause of infectious diseases. The application of the scientific method in medicine brought an end to unsubstantiated claims that considered a disease as the expression of cosmic imbalances, part of ancient practices like Alchemy and Astrology.
Today, software development is not that different from those dark ages of medicine. Our craft and skills are mostly acquired by oral tradition where learning is done through mentorship. Our remedies lack the precision of typical medical treatments that are measured in scientific units to prevent deadly repercussions. Yet, we wonder, why so many software projects fail, and why the adoption of improv methodologies, like Agile, fail to address the increasingly complex problems of a science that has become a crucial component of society.
My invitation is to read this post with an open mind. Some of the analogies here presented will seem exaggerated, others speculative, and some even uninformed and false. However, my intention is not to say something new, nor even lead a transformation in how software is developed. I would rather like to invite the reader to reflect on what we can do as software developers to improve in our craft, pay back the trust that society has deposited in us, and elevate our profession.
2. Background
I was the most surprised when first named Tech Lead. I certainly didn’t receive any training, nor did I requested or discussed the promotion. Is not that I didn’t want it, but I was certainly not prepared to manage a whole team from day one. The same happened as I first became a developer, proudly self-taught, my education consisted of online courses, algorithmic exercises, and online forums. It was up to me, and the patience of former bosses and customers, to learn how to design a database, maintain a Linux server, and craft user-friendly experiences.
Today as a software consultant, I face new challenges: I need to quickly assess the health of a codebase, emmit eloquent recommendations to quickly fix problems, and upskill or improve the productivity of in-house teams. The skills that I learned as a Tech Lead like documenting, designing, or managing projects seem comparatively inefficient to my new hourly rate. As I look back and evaluate my recommendations, I see how limited the arsenal of tools at my disposal truly is, how unfeasible is to clearly communicate years of experience to non-technical stakeholders, and yet, as I contemplate my limitations I also see the other side: the drastic improvement in productivity that good leadership creates.
During the last months, I searched for answers in books that included clean architecture, project management, systems design, consulting, and best practices in leading tech organizations but I did not found the answer I was looking for: how to quickly turn around productivity in small organizations. In despair, I look for inspiration in other disciplines like architecture, and eventually, I was fortunate enough to find the principles of this answer in the medical science of Pathology.
3. Eight Concepts to know about Medical Pathology
MIT has a program for non-medical practitioners called GEMS to give an outsider introduction to the medical profession. The first subject, and the only mandatory, is Principles of Pathology based on the homologous book by Robbins & Cotran. The next section includes potential applications to software development for the concepts presented in the first 7 chapters of the book. As a disclaimer, my experience in healthcare is limited to 2 months of research, short professional interactions as a consultant, and previous knowledge from past writings also inspired in life sciences [1][2][3].
3.1. The human body is the epitome of a well-engineered system
It is futile attempting to describe the level of sophistication of the human body. But I will give you two examples to get a grasp about it:
- Each cell holds about 750MB of data in its DNA (about the same as a legacy CD) and we have about 1 trillion cells in our bodies. In total, we store roughly a Zettabyte of data: the same amount that we created in our full history as a species up to 2016!
- Our brains have100 billion neurons, about the same number of stars in our galaxy, and each neuron has on average 1,000 connections with each other. As a civilization, humankind has yet to reach 8 billion people, and on average the 2+ billion Facebook users only have 300 “friends” each. Arguably, a single brain is a more complex network than our society as a whole by a factor of at least 25x.
As a Software Engineer, I try to approach the study and appreciation of the human body with humbleness. My opinion is that every problem worth solving has already been solved by nature.
3.2. Cells are the basis of human health
Diseases in the human body are the result of many individual sick cells. In retrospect, this seems obvious but as a developer, it ain’t, as we fail to articulate what constitutes a healthy software system.
Following the analogy, we might say that health in a software system (or lack of technical debt) is the result of well-written individual functions (or methods). Adopting the function as the basis of health enables us to empirically determine parameters for how a healthy function looks. As suggested in clean code practices this might include:
- The number of lines code, variables, and even tokens or characters.
- How clear the function and its variables are named.
- The number of input parameters and return types.
- Cyclomatic complexity (number of dependencies).
- The number of conditionals, iterations, or recursions used.
- Clean error and exception handling.
- How and where is the function invoked.
By defining how a healthy function looks we can start to approach the evaluation of a system as the aggregate health of its individual components.
3.3. Cells die due to injury, failure to adapt, or age
Here, the first interesting concept is aging: cells are programmed to die. But we hardly think of functions that way. We like to believe our code will be used ad infinitum. But imagine if a function would be programmed to be deprecated? The original developer would feel more comfortable knowing how much care to dedicate to different portions of the codebase.
Even more interesting, is that cells reproduce by mitosis (splitting by half and creating two copies) but the number of divisions is also limited to 40 to 60 times [4]. Consider how cleaner the code will be if a function would deprecate after a fixed number of modifications as measured by pull requests. We know that nothing harms more the readability and consistency of a codebase than unexpected shifts in product requirements.
Thinking about stem cells, that can adapt into any type of human cell, seems far. However, I can envision creating repositories of template functions that developers would clone and adapt to their needs. These functions will have best practices implemented to them, and their deviation from them would constitute technical debt.
The second interesting concept related is that cell death is also a consequence of failure to adapt. Software projects also often fail due to failure to adapt to market conditions.
This leads me to think that software development is a human activity unlikely every other one done in the past. Consider an artist, that after finishing their work, it remains intact. Or more traditional branches of engineering like civil, mechanical, chemical, or aeronautical: their designs often meet circumstances well-known in advance.
Even scientists, that aim to formulate theories, mostly assume the invariability of the laws that govern what they are trying to understand. Software is different perhaps a little bit like social sciences or business they find an array of unpredictable circumstances derived from human behavior.
In my opinion, the emancipation of software from business is one of its greatest curses, and we need to find a new word to define the process of producing software to create products that meet changing business conditions.
3.4. Inflammation is the mechanism for cells to heal
When you cut and blood, the skin turns redder as it heals. During flu, you cough and get fever as the body fights the virus. Both are symptoms of inflammation the mechanism to bring extra blood and white blood cells (leukocytes), to the damaged area.
How does healing occurs in a Software system? By bringing extra resources to the afflicted area. When a bug is found, developers rush to fix it, and after the solution is pushed it leaves a trail in the Git history. Curiously, the inflicted area is also stained in red as a fix usually involves more deleted lines of code.
Bug fixing leaves a second sign after is fixed, it is the absence of work in other areas as more developers' attention was entertained. As a result, the overall productivity of the team plummets. As the science of Software Pathology is discovered, software consultants will use the Git history to diagnose which areas have been inflamed
3.5. Blood (or its lack of) is the biggest source of cell disease
Everything that a cell has or needs is provisioned by blood. That is why cardiovascular diseases are the leading cause of deaths globally at 18% [5]. If a cell stops receiving oxygen or nutrients its health will deteriorate, and we need exercise to keep the heart strong and flux blood to the entire body.
A function receives data, and if it is of low quality, arriving with errors, or overly complex: the function will suffer. Thus, data hygiene maintains the codebase healthy.
The circulatory system is impressive in its own right: it stretches more than 66,000 thousand miles, with every cell carrying more than 1 million molecules of oxygen. On average, the heart pumps 2.5 billion times during a human life [6].
Software systems also have a heart: the database. Evaluating how data is queried, and fetched is the first step of diagnosing the health of a software system. How data flows through a system is probably the first diagram I would advise, to understand a system.
3.6. A surprising number of diseases can be attributed to genes
Before my research, I used to consider genes as static, highly reliable, exhibiting a low degree of variance. However, I learned that almost 70% of humans suffer from diseases attributed to genetic disorders.
The concept of genetics is foreign and difficult to translate to Software. Fortunately, through my experience, I’ve started, and participated in a fair amount of different projects. And I could argue that a portion of software projects are doomed from the beginning.
In my experience, most projects fail due to business reasons, and after several iterations, these projects have accumulated so much technical debt that continuing development is unaffordable. When thinking about the software we hardly stop to think of the history of a project: not about the development but about the people that participated in it.
A source code is the reflection of the decisions, desires, limitations, and experiences of the people that created it. As we study the health of software systems we need to understand how human behavior impacts the outcome of a project.
Either because a key member left, there is a difference in vision, agile practices are clumsily applied, or there is no congruent business model. As a consultant, I’ve learned that not every software problem lies in the code.
3.7. We have an immune system to heal ourselves
Self-healing systems have been increasingly part of the SREs toolbox: complex canary configurations in Kubernetes clusters, a predominant acceptance in the benefits of testing, or the automation of security code reviews. The concept of an immune system in software projects is closer to the reality that we might think.
The immune system is divided into innate and adaptative. The innate has learned from diseases and pathogens that have accompanied us during our evolution. While the adaptative learns from comparatively shorter periods.
Probably the next step in self-healing systems is preventing problems before they arrive (or show symptoms). This could happen as soon as we are developing, after establishing the rules of what quality software is, we can use AI copilot tools for advanced linting signaling problems as soon as they are typed.
Even better, as we trace back the causes of problems in software to product development, potential problems could be spotted as soon as they are assigned to developers. Having practiced BDD principles I believe that by using passing criteria the goal is not as distant as it seems.
3.8. Histology is the study of tissues or how cells are connected
I want to share the last concept to show how deep the understanding of medical diagnosis goes. After determining that cells are the basis of health, and carefully classifying each type of cells based on its function. Physicians have studied how cells are connected through tissues linking cellular biology with anatomy.
In software, we need to start thinking about the graph topology of how functions are connected. Contrary to the human body where the connections between cells are tridimensional, in software, functions interactions can become much more complicated. Because a single dependency can be shared by the two far away modules creating turmoil.
The knowledge of histology has been harvested through the meticulous examination of different cells and tissues using microscopes, staining, and freezing techniques. We need a similar academic and scientific effort to harvest the knowledge about software pathologies.
We like to think that Software is complicated, vast, and unintelligible. Full of different technologies, languages, and cloud providers. However, compared to the diversity in nature, the complex taxonomy that originated from evolution, and the humble tools that early scientists like Aristotle had at their disposal: the task of classifying software systems is comparatively simple.
4. Diagnose the health of a React Application
During the next section, I want to imagine how the job of a consultant would look following the analogy of a doctor and a patient. The tools that they would require, and the framework to approach problems with the precision that is expected from doctors.
I start exemplifying three distinct analyses a consultant could request from a codebase. Then, I will focus on the interactions with stakeholders to communicate the diagnosis and suggested treatments. The goal is to approach problems in software with quantifiable metrics to make data-driven decisions.
For this hypothetical exercise, I will use the opinionated architecture of a React application because it follows a functional paradigm, with a unidirectional flow of data and reusable components.
4.1. Measure the health of React Components
The first analysis a software consultant would request when approaching a new React application would be to measure the relative health of components based on some of the rules highlighted in section 3.2. which include the length in lines of code of a function (or component), the number of dependencies, error handling, naming, etc.
As an example of how this analysis would look consider the SCC GitHub package that measures lines of code, estimated cost, and the complexity of a project. Written in go, the package analyzes the source code in a matter of seconds:
While doubters might be skeptical, of how effectively this can be measured. The process would consist of aggregating data from open-source projects. Relating different metrics like Github issues, pull requests, labels, and stars correlations could start to appear. Finally, machine models could predict the optimal parameters for lines of code, component inputs, depth of interfaces, and many others for a React project.
4.2. Map the flow of data through components
After figuring out how healthy individual components are, we would like to know how those components are located and related to each other. We could use traditional flow diagrams to map how data flows through components. And like a river amid mountains, React applications are pyramidal architectures where data flows downwards to the user.
We could start detecting if some components are saturated with data, or maybe in others data travel a long journey before reaching its destination. Or using other diagrams we could also spot the components that have the most dependants or because of the high number of bugs detected they are the weakest links in the system.
My ideal flow chart would have these 3 features:
- Arrows connecting two nodes represents data flowing from one component to the other.
- It is possible to see the interface of the data between connections.
- The components are colored by health.
Physicians use x-rays, magnetic resonance imaging, and ultrasound to visually diagnose patients. Upon reflection, I’m surprised how underestimated spatial techniques are in software. Having a global representation of the system enables more accurate diagnosis and communication with stakeholders.
4.3. Review the short-term history of Git commits
The final analysis I would request is to understand where developers’ attention and resources were allocated during the last weeks. The chart I would propose is a stacked area chart where each color represents a different section of the code:
As an only change, I would map the area under the curve as it is done in Github insights to represent code deletions. This analysis would bring me two key pieces of information:
This analysis would bring me two key pieces of information:
- How productivity compares across different weeks.
These 3 analyses are only a subsample of the many possible a software consultant, tech lead, or CTO could request. It’s exciting to think that the analyses could be fully automated. And only after data about a project is harvested does the real consulting starts; where creativity, experience, and elegant design makes the difference.
4.4. Collaborate to emit a diagnosis of the problem
Going back to the patient-doctor analogy, the next step after the analysis and tests are performed is to emit a diagnosis. For doctors, how the diagnosis is communicated, is just as important to what is communicated. A full diagnosis should include:
- A transparent description of what the disease is.
2. An explanation of the causes and symptoms that triggered the diagnosis.
3. A map of potential outcomes based on the diagnosis.
Before proceeding with the treatment, patients need to understand and accept the diagnosis. Thus, trust becomes an essential component in the relation. There is a human component as the doctor communicates good or bad news, warnings, or words of fate. Example summarized diagnostic could look like these:
- New feature specifications are ambiguous, and the expected outcomes are not clearly communicated. This is because engineering does not participate at the product planning stage nor do they know about the business needs that prompt the requests. As a result, there is a lack of ownership by developers and no commitment to results.
- The new development is blocked by shared dependencies across multiple applications. Since the dependencies are not thoroughly tested, and no single person understands the repercussions of modifying them there is no certainty as to what will happen when changes are made.
- Bugs are taking too long to fix because monitoring is done manually. Additionally, logging is often insufficient and developers take time to reproduce bugs. Because the platform is highly configurable it’s not cheap to fully automate testing causing bugs to be discovered in production.
Note, how none of these diagnoses offer a solution, they are explained in plain language making it possible for non-technical founders to understand and agree with the diagnosis. Additionally. the diagnosis could be supported by data or precise examples that illustrate the problems and increase its deliverability. Finally, they are obviously aligned to the problems the client sees, and the outcomes they would like to achieve.
4.5. Recommend a treatment
Something that doctors don’t do is to make decisions for the patient. Instead, they only offer the alternatives available for them, possibly adding their personal recommendation. Just as we did with the diagnosis there are hints that I learned from doctors:
- If the problem is not urgent, there is no need to rush to offer a solution. Let the client digest the news about the diagnosis before proposing a solution.
- The least radical treatments should be communicated first. The more severe the diagnosis, the more human the communication needs to be.
- Use empathy and let the client know that you understand what they are going through.
- When making a recommendation, be sure to also communicate the risks, implications, and what they can do to improve the odds of success.
- Optimize for trust, there is little chance of advancing to a positive outcome if the client does not trust your judgment. And if possible, prioritize the client’s needs above your own: a doctor is all about personal sacrifice.
Ideally, the efficiency of treatments and recommendations are based on data. As software engineers, we are used to applying logic to solve our problems. Unfortunately, humans are not great to instinctively deal with probabilistic scenarios. And by trying to apply logic to complex problems we risk that cognitive bias clouding our judgment.
5. Turning Software Pathology into a Science
Medicine was not always scientific, in the past dubious remedies like bleeding or trepanation were used despite their limited efficiency, often worsening injuries. There is a possibility that future software scientists will look with the same astonishment our approach to object-oriented design patterns which could appear Ptolomeic efforts to impose a worldview into the observable data.
As a nascent science, there are 3 steps software development unequivocally needs to take to adopt the scientific method, settle opinionated debates, and institutionalize its practice. Else, we are left with dialectics, power dynamics, and mysticism to guide development.
5.1. A lexicon of diseases
“Flu” describes a well-known set of medical symptoms, causes, and consequences. The same happens for malaria, smallpox, cancer, and many unfortunate other diseases. At the same time, these names are based on taxonomies shared across life sciences that started with the classification of species in biology using Latin etymologies.
Having a precise vocabulary would enable us to communicate, think, teach and solve problems. And by avoiding using the same term to refer to multiple problems we could stop dogmas and beliefs that there is a one-solution that fits all, such as TDD (test-driven development), or DevOps (automation at all costs).
Before this research, I thought that the labor of classification done in other sciences was dull and unnecessary. However, only after understanding the role, it serves as a foundation, can I appreciate the activity in its full complexity and purpose. Today, we may think that classifying animals between vertebrates and invertebrates is evident. Similarly, for the respective animal kingdoms: birds, mammals, insects, etc. However, I now wonder: how many iterations did those classifications go through? And more importantly, how will a taxonomy of software look like?
My intuition tells me that just as biology followed a classification based on the evolution of life: software should as well follow a historical taxonomy. Thus, the map software pathologies would require to trace back how new technologies also engendered new bugs and complications. Neither should we forget, that software is a social activity and one cannot compile a lexicon of maladies excluding the human element.
5.2. Understanding of causality
Autopsies were a breakthrough for medicine: they enabled physicians to examine the causes of death, learn about anatomy, validate hypotheses, and settled discussions. Today, we perceive irrational that religious impositions delayed medical progress.
Currently, the practice of project dissections and post-mortems is discouraged either because of intellectual property or business motivations. Yet, if we would like to decrease the failure rate of software projects we need to understand the reasons that cause projects to fail. Embracing established academic practices like publishing papers and designing experiments.
Causality is the link between diseases and treatments.
5.3. A plethora of treatments
Often, it is not possible, to determine a priori whether a particular treatment will succeed or not. In medicine, new drugs go through rigorous tests using laboratories, animals, and control groups before being offered to the public. Even after being released, many drugs are closely guarded by doctors and institutions.
I’m not advocating for the same level of regulation in software, but I do admire the scientific rigor. Today, I am aware of how limited my experience is compared to the spectrum of possibilities faced in software projects. Especially, considering the ever-changing business and technological landscape. At this stage of my career, I’m particularly concerned about how expensive it has become to acquire more experience by testing new solutions. Today, as I’m often responsible for teams and projects: exploring new solutions seems increasingly costlier and riskier.
As a consultant, I hope that in the future, I’m able to access well-written, exhaustive case studies before proposing a solution or treatment to a customer.
6. Conclusion
I would like to make a welcoming invitation for the reader to share their impressions, and hopefully continue with the work proposed in this essay. For me, there are 3 clear directions to go from here:
- Label the different circumstances faced in software projects.
- Develop automated analysis to diagnose codebases.
- Publicly document post-mortems and success stories.
As a personal opinion, I think that developers are among the smartest people in society, and there is an unspoken belief that we will figure things out. However, as time has passed, and I’ve gained experience, the only thing that I’ve certainly learned is to remain humble and always look to keep learning.
Finally, I feel optimistic as I now know what the tools and frameworks I need as a consultant. And more importantly, I’m excited to see what changes will be brought to the industry during the next 50 years, when I retire.
7. Acknowledgements
There is no great story without risk, and this time, I want to thank the people that connect with me through my writings. In particular, Brett from software.com, who reached out and shared the vision from an earlier post I wrote when I first became Tech Lead: understanding is one of the greatest gifts you can make, and a big encouragement to continue as a writer.
I also like to share appreciation with my clients, that tolerated my learning curve, posed interesting challenges, and allowed me to discover a new passion: helping people. And above all, to the people I love that accompanied me during my research and have done so for the last 2 years: my heart is with you.
Finally, I also want to thank you: the reader. Thank you for following me, for your claps, and hopefully for sharing this story. Most likely, this was not an easy read but if you want to read more from me, I recommend you my startup trilogy: The Viral Startup, A Post Covid World, The Laws of Growth. And feel free to share your thoughts, ideas and connect with me.
Warmest regards,
Santiago M.
