Archive

Coding

Making Tomorrow’s Big Balls of Mud Today

What is a Big Ball of Mud?

In software development, the term “Big Ball of Mud” refers to a system or codebase that has become so tangled, convoluted, and disorganised over time that it becomes increasingly difficult to maintain, modify, or understand. It’s a metaphor for a software product development that started with good intentions but gradually deteriorated into an unstructured mess due to a lack of proper planning, design, and adherence to best practices.

Consequences

The consequences of a Big Ball of Mud can be severe. It hinders productivity, increases technical debt, screws with predictability and schedules, and makes it challenging to introduce new features or fix bugs. Developers often find themselves spending more time trying to understand the existing code than actually writing new code. This can lead to frustration, decreased morale, and a higher risk of introducing further issues.

The Rise of AI-Centric Coding

And a paradigm shift is looming on the horizon – a transition towards AI writing code – and primarily for artificial intelligence (AI) readability and maintainability. While human-readable code has long been the desirable approach, the remarkable advancements in AI technology necessitate a reevaluation of our coding practices and the use of Ai to write code to harness the full potential of these sophisticated tools.

As AI systems become increasingly integrated into software development workflows, the need for code that caters to AIs’ unique strengths becomes paramount. This shift will give rise to coding styles specifically tailored for AI readability and maintainability, encompassing the following characteristics:

Abstraction and Modularisation Paramount

AI systems thrive on highly modularised and abstracted code, where individual components are clearly separated and encapsulated. This coding style will emphasise smaller, self-contained units of code with well-defined interfaces, promoting better organisation and encapsulation, aligning with the strengths of AI systems.

Formalised and Explicit Syntax

In contrast to the conventions and implicit understandings often relied upon by human programmers, AI systems will benefit from a more formalised and explicit syntax. This could involve additional annotations or metadata that make the semantics of the code unambiguous and readily interpretable by AI systems.

Pattern Recognition Optimisation

AI systems excel at recognising patterns, and the coding style will be optimised for this strength. Consistent naming conventions, structural similarities, and other patterns that can be easily recognised by AI systems will become more prevalent, enabling efficient pattern recognition and analysis.

Reduced Redundancy (DRY)

AI systems are better equipped to handle and maintain code with minimal redundancy, leading to a coding style that emphasises code reuse, shared libraries, and other techniques to reduce duplication. This approach will not only cater to AI systems’ strengths but also promote code maintainability and efficiency.

AI-Tailored Documentation

Traditional human-readable documentation and comments may become obsolete in an AI-centric coding paradigm. Instead, the emphasis will shift towards creating self-documenting code that can be seamlessly interpreted and maintained by AI systems. This could involve incorporating structured annotations, metadata, and other machine-readable elements directly into the codebase.

The documentation process itself could be automated, with AI algorithms capable of parsing the code structure, analysing the annotations, and generating comprehensive documentation tailored specifically for AI comprehension. This documentation would be optimised for pattern recognition, logical inference, and other capabilities that AI systems excel at, ensuring that it remains up-to-date and consistent with the evolving codebase.

AI-Generated Code for Machine Consumption

Furthermore, the advancement of AI technology raises the intriguing possibility of AI systems themselves generating code in a style optimised for machine consumption, rather than human readability. This AI-generated code could forgo traditional conventions and practices aimed at enhancing readability for human developers, instead favouring structures and patterns that are more readily interpretable and maintainable by AI systems themselves.

Such AI-generated code might be highly compact, with minimal redundancy and a heavy reliance on abstraction and modularisation. It could incorporate complex mathematical models, advanced algorithms, and unconventional coding techniques that leverage the strengths of AI systems while potentially sacrificing human comprehensibility.

As AI systems become increasingly integrated into the software development lifecycle, they could potentially maintain and evolve this AI-generated code autonomously, with minimal human intervention. This paradigm shift could lead to a scenario where the primary consumers and maintainers of code are AI systems themselves, rather than human developers.

Factors Contributing to Big Balls of Mud

While embracing AI-centric coding practices offers numerous advantages, we might choose to be mindful of the potential pitfalls that could lead to the creation of ‘big balls of mud’ – tangled, convoluted, and disorganised AI-generated codebases that become increasingly difficult to maintain and modify.

Today’s Factors

In the current software development landscape, where human readability and maintainability are still the primary focus, several factors contribute to the formation of big balls of mud:

  1. Lack of Architectural Foresight: The absence of a well-defined software architecture from the outset can quickly lead to a patchwork of disparate components, hindering maintainability and coherence.
  2. Prioritising Speed over Quality: The pursuit of rapid development and tight deadlines may result in sacrificing code quality, maintainability, and adherence to best practices, accumulating technical debt over time.
  3. Siloed Development Teams: Lack of coordination and communication between teams working on the same codebase can lead to inconsistencies, duplicated efforts, and a lack of cohesion.
  4. Lack of Documentation and Knowledge Sharing: Inadequate documentation and poor knowledge-sharing practices can make it challenging for new team members to understand and maintain the codebase, exacerbating the tangled nature over time.

Future Factors with AI-Driven Development

As we transition towards AI-driven software development, new factors may contribute to the metastasizing of big balls of mud, if not appropriately addressed:

  1. Not instructing AI to include AI-friendly code generation and the needs of AI vis codebase readability and maintainability. Prompt engineeres in the code generation space take note!
  2. Lack of AI Training and Optimisation: Without proper training and optimisation of AI models for code generation and maintenance, the resulting codebase may lack coherence, structure, and adherence to best practices.
  3. Inadequate Human Oversight and Understanding: An over-reliance on AI without sufficient human oversight and understanding can lead to opaque, difficult-to-maintain code that deviates from architectural principles and design patterns.
  4. Inconsistent AI Models and Tooling: Using multiple AI models and tools for code generation and maintenance without proper integration and consistency can lead to fragmented and incompatible code snippets, exacerbating the tangled nature of the codebase.
  5. Prioritising Speed over Quality and Maintainability: Even with AI-assisted development, the pursuit of rapid development and meeting tight deadlines at the expense of code quality, maintainability, and adherence to best practices can lead to long-term technical debt.
  6. Lack of Documentation and Knowledge Sharing: Inadequate documentation and poor knowledge-sharing practices can hinder the effective use and maintenance of AI-generated code, making it challenging to understand the context, design decisions, and rationale behind the code.

By addressing these factors proactively, software development teams and organisations can harness the power of AI while mitigating the risk of creating tomorrow’s big balls of mud, ensuring that codebases remain maintainable, scalable, and aligned with inhouse best practices.

Conclusion

The future of coding lies in embracing the capabilities of AI systems and adapting our practices to harness their full potential. By prioritising AI readability and maintainability, we can unlock new avenues for efficient and optimised code generation, enhanced collaboration between human developers and AI systems, and ultimately, more robust and scalable software solutions.

While this transition challenges traditional assumptions and beliefs and invites a major paradigm shift, it is an exciting prospect that will revolutionise the software development industry. As we navigate this paradigm shift, it is essential to strike a balance between leveraging the strengths of AI systems and maintaining a level of human oversight and understanding, ensuring that our code remains accessible, maintainable, and aligned with the evolving needs of the host business.

 

Who Cares How We Code?

The Premise

As developers, we’re a smart bunch. We know our stuff and can generally be trusted to choose the best approach to getting the job done, right? After all, the goal is to get that program up and running in production as quickly as possible. What could possibly go wrong if we cut a few corners here and there? A bit of spaghetti code never hurt anyone. And technical debt is a conscious and intentional choice, yes?

The Bitter Truth

Sadly, this cavalier attitude towards development practices is a recipe for disaster further down the line. While it may seem like a shortcut to production Heaven, it’s more akin to paving the way to Maintenance Hell – and Future Costs City. Let’s explore why we might choose to actually care about how we code.

Compromising Schedule Predictability

Messy codebases compromise team’s ability to predict how long something is going to take. The more the mess, the more unreliable the schedule.

The Future Payback Trap

We can compare writing sloppy, unmaintainable code as analogous to racking up maxed-out credit cards. It’s taking on inevitable future payback down the line, and just like financial debt, it accrues “interest” in the form of extra development costs that compound over time. That once-scrappy codebase becomes an ungovernable mess that’s exponentially harder to change, optimise, or extend. Before we know it, we’re spending more time untangling our own spaghettic nightmares than making meaningful progress.

The Collaboration Conundrum

In most cases, a codebase is a team effort across its lifetime. If you don’t maintain a minimum level of quality, good luck onboarding new team members or even having your future self make sense of the tangle a few months down the road. Sloppy code breeds knowledge silos and cripples effective collaboration.

The Debugging Debacle

Well-structured, self-documenting code that follows good architectural principles makes it infinitely easier to debug issues and safely update the software over time. In contrast, a patched-together “codic dervish” is virtually impossible to decipher or modify without potentially disastrous unintended consequences.

The Performance Pitfall

While your hacky script may seem to work for that small prototype or MVP, codebases that cut corners on fundamental coding practices and design patterns simply won’t be able to scale gracefully as usage and complexity grow over time. Code quality is paramount for managing performance under load.

The Futility of Quality Assurance

When we don’t make code quality a priority from the get-go, good luck getting meaningful code reviews or implementing a robust quality assurance approach. Code reviews become an exercise in futility, and QA turns into a fruitless game of DevOps whack-a-mole, constantly putting out fires in an inherently unstable, unpredictable product.

The Craftsmanship Principle

At the end of the day, consistently writing clean, maintainable code is one of the hallmarks of competence, as opposed to a mere hack. By treating our craft with care and prioritising technical excellence, we’re investing in the long-term success of our products, our teams, and our careers. But who cares about the long term?

The Creative Developer: Coding is Just Our Medium

How many software developers when asked what they do for a living reply “writing software”? Just about 100%, I’d guess. The very title of “software developer” implies we spend our days pounding out code, line after line of instructions for computers.

But is that truly an accurate picture? I would argue that the analogy of “writing” software promotes some problematic assumptions. It focuses purely on the technical aspect of coding, ignoring all the other important facets of bringing software to life. It perpetuates stereotypes of programmers as nerdy code monkeys, heads down in front of a keyboard all day. And it fails to capture the deeply creative process that software development entails at its best.

In reality, we developers don’t just “write” software – we attend to folks’ needs, crafting systems, experiences, solutions and above all, interpersonal connections. We collaborate, gather requirements, make trade-off decisions. We envision how people will interact with the products we craft. Code is simply our medium for bringing strategy and creativity to life.

Software development has as much in common with engineering, architecture or even storytelling as it does with coding. There is an artistry and imagination behind truly great tech-based products that goes far beyond syntax. The attendants of the future will be at least as fluent in humanities as mathematics or computer science.

So the next time someone asks what you do, don’t reflexively say you “write” software. Share how you attend to users’ needs, strategise solutions, and creatively work with teammates. Let’s put to rest the tired stereotype that developers are code-writing scribes! What we do entails far more multi-dimensional and meaningful attending to needs, products and people.

Girls Who Don’t Code

Girls and women are ideally placed to become real developers (by my definition*) and yet they want to CODE?

*My definition:

A real solutions developer is not so much someone who possesses technical expertise, but rather has the ability to connect with people and truly understand their needs.This requires a high level of emotional intelligence and empathy, as well as excellent communication and interpersonal skills. A real solutions developer builds relationships with clients, collaborates with team members, and creates solutions that meet the unique needs of each individual and group. By putting people first and prioritising human connections, a real solutions developer is able to deliver truly transformative solutions that make a difference in people’s lives.

See also: #NoSoftware

You’ve Got It All Backwards About Coding

Coding (as in programming) is, essentially, a form of structured note-taking. While it is true that computer programming enables machines to execute complex tasks, we may choose to recognise that it also serves as a powerful tool for humans to express their thoughts and ideas in a systematic manner. By employing a well-defined syntax and set of rules (albeit somewhat arcane), programming languages facilitate the clear and concise recording and communication of ideas, making it easier for individuals to plan, reason, and comprehend.

The act of coding allows programmers to break down complex human needs into smaller, manageable components. This structured approach not only makes it easier to understand and solve problems but also aids in the sharing of knowledge among peers. As a result, programming languages are not so much tools for instructing computers but also a means for human collaboration, fostering creativity and innovation.

Moreover, having a computer execute the code acts as a check on the utility of the programmer’s notes. This execution serves as a validation of the thought process, ensuring that the concepts and logic so encoded are sound and functional. By identifying errors or inefficiencies in the code, programmers are encouraged to refine their ideas, consequently improving the quality of their thoughts. Thus, code is not so much a set of instructions for machines, but a valuable tool for human expression, communication, and growth.

I’ve not called myself a software developer for at least thirty years. That’s not to say I’ve stopped coding. Far from it. But the end in mind has changed. From “developing software” to “attending to folks’ needs”. Seems to me that latter frame offers far more potential for satisfaction – both for me and for those I serve – than coding ever did. See also: #NoSoftware and the Antimatter Principle.

After all these years, I still love coding (as in writing software).

It’s just that it’s tainted by the certainty that there’s so many other more effective ways of adding value and meeting folks’ needs.

Spending time coding feels so… self-indulgent.

Scope of Ignorance

Most of the developers and development teams I used to work with when I was a software development consultant had a relatively narrow view of the skills and knowledge necessary to be “competent developers”. Here’s an illustrative graphic:

Generally, to make progress on improving things, and to earn the moniker of “software engineers”, a wider scope of skills and knowledge was necessary. Not only did these development teams lack this wider scope, they were both ignorant of the many additional areas of knowledge and resistant to learning about them. The common response was “What are all these strange topics, and NO WAY! do we need to know about them”:

Aside: Now I’m an Organisational Psychotherapist, their ignorance is no issue – and no stress – for me. They can learn or not learn in their own time. Progress is on them (and their higher-ups).

– Bob

Excolat in Pace

There’s a common idea which has been doing the rounds, ever since development (coding) first became a thing. We might sum it up as:

“Developers just want to develop in peace.”

As someone who spent more than a decade in the development trenches (and still does development today, occasionally), I can instantly relate to this issue. Indeed, focus is the key question. Any kind of interruption or distraction whilst reading or writing code can suddenly evaporate the evolving mental model of the inner workings of that code, a model built up painstakingly, with deep concentration, over twenty minutes or more. So three for four interruptions or distractions, however trivial, can wipe out an hour of otherwise productive effort. And that’s before we get to the question of frustration, the impact of frustration-induced stress on the individual, and the stress-related impairment of cognitive function more generally.

On the other hand, having developers separated from the folks that matter introduces other productivity-sapping dysfunctions, such as misunderstanding folks’ needs, building the wrong things, and reducing the joy of getting to see how the developers’ efforts make a difference to others.

Conundrum

So, how to ensure developers have the peace they need to focus intently on their coding efforts, whilst also ensuring they have sufficient interactions with the folks that matter – sufficient to ensure that needs are understood and the right solutions get built?

In the past, specialist intermediaries a.k.a. Business Analysts and Project Managers have served to address this conundrum. And solutions (including the role of specialists, and the workplace environment) have been imposed on developers without much consultation. Rarely have developers, or the folks that matter, been involved in finding a way forward together.

Personally, and in the context of self-managing teams in particular, I’m all for the teams and their customers (both internal and external) getting together and thrashing out a way forward. And then having regular check-ins to improve those ways of working together.

As an example, BDD (Behaviour-deriven development) is a current set of practices that offers one such way forward. Customers and suppliers sitting down regularly (as often as several times a day, for maybe twenty minutes at a time) and working through a User Story, Scenario, or Use Case, together.

And let’s not forget that the other folks involved, aside from the developers, also have their day jobs – jobs which require them to focus and spend time on things other than working with the developers.

How do you, your teams, and their folks that matter, propose to tackle this conundrum? How are you handling it at the
moment?

– Bob

Postscript

Another option that comes to mind is mob programming a.k.a. mobbing, particularly with the involvement of folks having in-depth domain knowledge (i.e. customers and users).

The Future of Coding Environments

How would Scotty or Geordie go about writing code for the Starship Enterprise? Would they write code at all? Would they just interact with the Computer via speech or holodeck, or would a keyboard of some sort still have a place? 

In any case, my interests have always stretched beyond matters of organisational effectiveness, beyond matters of human and humane relationships, and beyond matters of how the work of software and product development might better work.

One of my other abiding interests has been the nature of programming. Indeed I spent more than two years, decades ago, on conceiving, designing and implementing a proof of concept for the kind of development environment I’d like to use myself, when writing code. At the time, the work was codenamed “Simplicity”.

My core feature set / wish list includes: 

  • Editing source code directly in the AST, rather than editing source code in text files
  • Direct and incremental compilation of source code as it’s being entered
  • Multiple coders editing in the same AST concurrently
  • Live editing of the AST “in production” (with appropriate safeguards built-in)
  • One homogenous AST for each entire (live production) system
  • Source code control / version control features built right in (and automated away from distracting the coders)

OK, so this may not be the kind of development environment Scotty or Geordie would recognise. But it’s a world away from all the crap we have to put up with today.

Blockers

So why don’t we see more movement towards the emergence of some of these features in our development environments today? In a word: conservatism. Developers en masse seem disinclined – or unable – to look anew at their tools, and dream.

“The future is a foreign country; they do things differently there.”

– Bob

Further Reading

The Mjølner Environment ~ Görel Hedin, Boris Magnusson