The Master Developer's Path of Productivity
Doing more with less, through self-awareness and incremental improvement.
In this Tech Talk given at GenUI in May 2019, Shane shared the insights into developer productivity he has gained from decades of mastery.
GenUI Software Architect Shane Brinkman-Davis Delamore has spent his career solving problems in developer productivity, driving at the essence of what we are really doing when we create software products, and how to make our special kind of knowledge work more efficient and enjoyable.
“Throughout your life advance daily, becoming more skillful than yesterday, more skillful than today. This is never-ending.” -Yamamoto Tsunetomo (1659-1719), Hagakure
I studied Japanese in college and one phrase that stuck with me was: “chiri mo tsumoreba yama to naru” which translates to, “Even dust, if piled up, can become a mountain.” Mastery is the result of small, incremental, and directed improvements over time. I want to share some ideas on how you, throughout the normal workday, you can make these incremental improvements and take steps on the path toward mastery.
Counting my teenage years, I have been a developer for almost three decades. I can offer some concrete insights, learned over those years, but most of all, I offer you a philosophy on how to be a self-improving developer. And, while I’m addressing developers, I do think much of this applies to designers – in both cases, it’s about creating something new that has never been seen before.
As developers, we’re delivering software artifacts to customers. The major cost is the developer-hours it takes to create the software. We can define “productivity” by answering two related questions: How much value will we deliver? And how long will it take?
The developer role requires a multitude of choices, even if designs were provided up front. We make choices every moment we’re writing code that, in turn, affect both the final value delivered and the amount of time it takes.
There can be a huge amount of difference in performance between developers. Every single developer has the ability to improve their skill and increase their productivity many-fold. As explored in an article by Yevgeniy Birkman, the numbers suggest that 10x value-per-hour-worked is not even the ceiling of how you, as a developer, can increase your output. I’ve been obsessed with this concept since I was a teen, first getting into programming. I thought, “I can build anything I ever wanted!” I soon realized that I didn’t have the time to build everything. There was so much I wanted to build, so I started to care about producing as much as possible within the hours available. In other words, I got obsessed with my own, personal productivity.
The first half of productivity is “doing more.” I think the key to doing more is separating the “essence” from the “artifact.”
The first, and most important step, is to try to understand the essential problem that the client is trying to solve. You’ll often have to read between the lines, because it’s a rare client that truly understands their problem. Usually, there is just one specific, focused need whose solution ultimately determines the success or failure of a project. Most of the other perceived needs are usually just noise. If you nail that one, core need, not only will you succeed in making the client happy, but you’ll do so with a smaller, better, and easier to maintain product because you eliminated the non-essential, artificial requirements.
“Is this actually the right solution, or am I just doing it because everybody else is?”
Only once you’ve discovered the essential problem, can you start to look for its essential solution. What solution will fully solve my client’s problem with minimum time, effort, and complexity? An essential solution is timeless, simple, powerful, adaptable, and most of all, focused. This certainly applies to spec gathering and design work but, ultimately, it comes down to the developer. You’re writing code and have to make decisions that require you to understand the essential problem and its essential solution. This is one part of leveling up. We’re always looking to deepen the understanding of our client’s problems while broadening our knowledge of possible solutions.
While you are searching for an essential solution stay aware of the trap of artificial, historical solutions. Just because a solution is standard, common or ”proven” doesn’t mean it’s appropriate for your problem. One of my weaknesses is that I don’t follow the status quo and am always going off on my own path. But the status quo can also lock you into solutions that may not be ideal. Often it’s important to take a step back and ask, “Is this actually the right solution, or am I just doing it because everybody else is?”
You may know the acronym Y.A.G.N.I. (You Ain’t Gonna Need It). It’s awkward to say. It’s inelegant. And it’s not the essential expression of the idea. Instead, I use Z.E.N. (Zero Extra Nuts). When you’re building something, don’t add anything more than you actually need in order to meet specs. Yes, this is hard to do because developers love to overbuild. But the reality is that, if you overbuild,
- it will probably be wrong for the specs;
- the specs will change and it will become irrelevant; or,
- it’ll end up as another piece of code that needs to be maintained without adding any real value.
I struggle mightily with this, but we must strive to build exactly to the specs - towards solving the essential problem and only the essential problem. By all means, push back if you think the specs are wrong. Always call that out. But if the specs are right then don’t overbuild. Just meet them as efficiently and elegantly as possible.
… With Less
The other half of productivity is doing things “with less.“ One of the things I learned early on is that you can ship any project on time, under budget, and with quality. Period. You just have to cut features. Often people don’t want to do that, but the real insight is that a product gets better as you remove features. You always think you’re giving up value by removing a feature but – and here’s why this matters – most of the time the core problem you’re solving is for one user story. There are many other little side user stories like log-in and authentication that aren’t about solving the user’s real issue. We think side features matter when in fact 90% of the customer’s time is spent on one path. If you can focus on making that one path optimal without distractions, then you arrive at a better product. Since we’re often not directly shipping product, we have to work with our customers to help them understand that less is more.
“You can ship any project on-time, under budget, and with quality … You just have to cut features.”
I want to emphasize that, with mastery, there is no plateau of ultimate achievement. You can always understand the essential problem more deeply and solve it with an ever-closer-to essential solution.
Solving Problems with Less Work
In addition to removing features, there are many other ways to produce the same value for less work. Many of them fall under the heading of being more “dry.“ D.R.Y. stands for Don’t Repeat Yourself. 90% of bad programming is repeating yourself. I have seen a myriad of ways that code goes wrong, and it’s usually due to “wet“ code. If the goal is delivering more value per hour, then you get there by doing less work. To the novice, copying code seems like a fast solution, but most developers quickly find out the woes of simply copying code. However, experienced developers still violate dry all the time. We may not literally copy code, but when we use the same pattern in multiple places, we are still producing wet code.
Less code is less to maintain, and duplicate code is especially perilous. Duplicate code breaks down because when you need to make changes, you have to update code in multiple places in perfect unison. Worse, you may not even know that it’s in multiple places and only update one and not the other. I obsess about eliminating duplicate information – not even just copied code but similar patterns throughout. At this point, I manage around 50 open source libraries because, if there’s a pattern I’ve done five times, then I tend to take that pattern, create some tests, and move it into its own, small, focused project. When I have updates, I only have to update and test it in one place.
Get annoyed with repetition. This is the key driver in my own improvement. From my earliest days, I became frustrated with repetition. If I do the same thing twice while on a job then I ask, “How can I automate this?” That’s the beauty of programming: you can automate anything.
One example is curly-brackets and parentheses. We type these all day long, struggling to indent them properly and keep them precisely matched. Even if you only spend an average of 10 seconds on each, after doing 100 of them in a day you’ve lost not only 20 minutes of time, but probably been yanked out of “flow” more than once to solve mismatchings. Little things like that add up and cost valuable time. Another way to think about it is you already indent your code, so curly-braces violate dry – you are redundantly encoding the block structure of your program twice-over. Auto-indent, bracket-insertion and bracket-matching tools help, but the best solution of all is when you make the problem impossible in the first place. That’s why I use indention-based languages like CaffeineScript.
“Get annoyed with repetition.”
Get obsessive with minute details. Syntax errors break you out of flow. This is an indication of a suboptimal workflow. All too often I find developers take the attitude of “suck it up” or even pride in their ability to solve these irrelevant problems. My mantra is: blame the tool, not yourself. The tool is failing you if it can’t figure out what you’re trying to do. Whenever you see a syntax error, take a moment and think about how you can either train yourself to avoid those errors in the future or how the tool could be redesigned to make the error impossible in the first place.
In addition to eliminating anything that yanks you out of flow, also note whenever you are waiting for something. Waiting for the CPU is lost time. Build times, load times, test times: these matter and it’s worthwhile to refactor periodically to make them fast.
Libraries that Reduce Repetition
These are three examples of open source libraries I’ve built to reduce repetition in my projects:
One of my key insights is: The names of files and directories are part of the source code. In most code-bases, the directory structure and file-names are duplicated in structures defined inside the source code files as well. Neptune-Namespaces attempts to DRY things up.
Art-Build-Configurator (npm) configures my package.json and other standard project files. It automates all of the little minutiae of package management. I need to do this because I’m managing dozens of open source libraries. Package.json in particular drives me a little crazy because I value “convention over configuration.” Everything should be code. You can’t even have comments in a package.json file! Most of package.json could be filled with “convention” – automatically generated with optional overrides. Art-build-configurator does just this.
Art-Testbench takes my namespace tree and generates a test suite hierarchy. My tests are already organized in a nice directory structure, why should I have to duplicate that information by also defining my test-suite hierarchy in code?
Writing Less Code
This is my main mantra. Given two alternate solutions which meet specs, the shorter one is almost always better. I don’t mean abbreviated names. Readability is important. I mean fewer tokens. A full variable name is a token. I use descriptive variable names while attempting to reduce the number of tokens overall. Over long-lived projects, I’ve found every line of code you write will, on average, be rewritten 10 times and be read 100 times. By committing a line of code to your source tree, you are committing to approximately 100x that time-cost through the lifetime of the project.
Code is a liability, not an asset. Companies think of code as an asset. Yet it’s the running application that delivers value to a customer. The code itself requires maintenance, and this burns dev time – the most expensive part of a project.
“Code is a liability, not an asset.”
Debugging is a Waste of Time
We inevitably spend a lot of our valuable time fixing bugs. Yet 95% of bugs are trivial to fix. They usually only require a few edits, but they are equally often difficult to find. Thus time wasted on debugging becomes a matter of visibility. It’s about being able to understand and see what your code is doing so that, when you run into a bug, you can find it and fix it quickly.
I have a mixed relationship with debuggers. They end up being so complicated that I often spend so much time dealing with their issues that it almost neutralizes their benefits. Yes, you can get really good at a debugger, but what I’ve found even more effective is building introspection tools into my code. Whenever I write a class I add tools that let me inspect it easily at runtime. Just like you build tests, it’s equally important to build introspection tools throughout your project in a standard way that anybody can use.
Two more features of my console are particularly useful. First, when I log an image, I get to see the actual image. I do a lot of UX work, and being able to see what an image is rather than just “it’s an image” is huge. Second: have you ever logged a “promise” in your console? Right. It’s useless. It also tells you nothing other than “it’s a promise.” More and more promises are appearing everywhere in our code, and yet logging one only displays a status akin to “not done yet.” This tells you zero. Worse, usually the promise is completed a millisecond later. Literally, one millisecond later. So to inspect one, you end up having to write custom code to try to capture it, then log it - a millisecond later. Whenever I log a promise to my custom console, it tells me it is “pending” and, as soon as it resolves, it also tells me the result. I can then log promises and inspect them easily. The console is smarter still in that you can give it any arbitrary JSON structure with promises littered throughout. It will wait until they’re all done to tell you what all the promises resolved to.
This is what it means to build powerful introspection tools into your code. You can go far beyond the meager help provided by standard debuggers. Your custom tools can be domain-specific. They can distill the runtime state into its essence, so you can quickly understand what’s happening, fix your bug, and move on. They can also be more general-purpose and apply project after project. There is so much opportunity for new, well-designed introspection tools.
Self-Awareness for Software Developers
I want to discuss devoting attention to your awareness practices as software engineers. Self-awareness, mindfulness, and other ideas from the self-care community do apply (and are wonderful for anyone) but I’ve ratcheted these concepts down to developer-specific insights. It all starts with observation.
Automate Workflow Tracking
Find tools to automate tracking and do this work for you. I use Timing on Mac. Throughout the day, it logs the apps I’m using for when and how long. It also logs the file I have open and the path that I’m in, or the URL, and offers many slice-and-dice ways to automatically categorize my day into “projects” as I specify. This is a great way to reflect on time spent. It can say, “You switched to the browser here five times.” If you went to Facebook, it actually tells you, “You went to Facebook five times for 18 minutes.” This is both very precise and effortless once setup.
Now, I don’t go back and obsess over it. Sometimes it helps to review how I spent my time in a previous week. I pull it up and look at where I devoted time and where I didn’t. “Was that a reasonable use of my time?” While the tool can be useful for billing, I mostly use it as a chance to reflect. It runs 24/7 on my computer so I can even go back years to access information about how my time was spent.
Intentionally Plan for Introspection
Dedicate time for introspection. If you don’t plan ahead then it isn’t going to happen. To help with that, I use a great reminder app on iOS called Due. Its key feature is it nags you until you complete the task. Most reminder apps remind you one time and, if you miss that reminder, the task is easily forgotten. With Due, the re-reminder interval is configurable to once an hour, every 10 minutes, or any duration you choose. I use it religiously and devote several times throughout the day to reminders to pause for introspection.
The main benefit is simple: the app reminds you about a task until you mark it done. So you actually get things done.
Take Notes… Constantly
Pick a note-tool that makes you happy and whenever you run into one of those little things that slow you down (like a syntax error) just jot a little note and go back to work. It doesn’t have to distract you more than, “I just wasted five minutes with this syntax error” before you return to coding. Then, when you return to your notes, you have the awareness to say, “Oh, I did that syntax error twenty-five times last week. Maybe there’s a way I can make that better?”
I strongly encourage upping your git commits by a factor of 10. I git commit every time tests pass. When you screw something up, being able to rollback to an earlier version of the day is a big deal. I rollback, forward, and compare with diffs quite often whenever I’m debugging. “It was working 10 minutes ago! What did I change?” It’s also a great reflection tool because, each time you commit, you’re putting in a little note explaining your changes. In addition to committing more often, you should also break up your commits. Make each the smallest, logical set of non-breaking changes you can. Not only does it give you more options if you need to roll-back, but it provides more documentation about what you did for you and your team.
There are many ways to nurture inspiration. I love finding cool open-source projects and cracking them apart to see how they work. In particular, look for opportunities to unpack the reasoning behind choices made and ask yourself what you would do differently. That mental exercise ups your game.
You’ve probably heard people say you should learn a new language every year or two. I think that’s very helpful. But it’s also worth trying to write a programming language yourself. Even a tiny project where you write a language over a few days can radically shift your perspective on programming languages and programming in general. If you use a good parsing library, it isn’t actually that hard to do. Yeah, I have a library for that, too: Caffeine-Eight (npm).
When I’m stuck, another one of my methods is physical: I go to a magazine stand and browse anything that grabs my eye. I could be grappling with a complex algorithmic problem, but I’ll let myself think things like “Wow, that’s a cool, red banner on that magazine!” Adding new elements to your mind, even if completely unreleated, refreshes the brain, allowing you to return to a project inspired and with the momentum to move forward. Nowadays, when you go to a magazine rack, it offers an overwhelming amount of stimuli. This is actually good. Let go of opinion and grab whatever stands out. Look at it. Let it in.
Self-Review and Organization
Ultimately, logging is only useful if you go back and review it. Some prefer an end of day self-review, but I’m not rigid about this. Just get the concept in place and take regular time for reflection. I self-review regularly and a key part of this process is organizing my notes. Go back periodically, particularly if you tag your notes in some way, and say, “Oh, syntax errors. Look at these trends over the past week. Is there anything here I can still do something about?” Most note tools let you edit old notes. Sometimes I edit to clean up typos, but I recommend leaving your notes mostly intact because they capture a slice of time. Instead, write a new note to summarize and reference the old ones.
Ultimately, think about ways you can waste a little less time during your day. That is how my CaffeineScript programming language was born. I was absolutely not going to write another programming language – I’d done a few. For a year, I threw notes into Evernote and got to about 200 until realizing, “I know exactly what this thing looks like and I need it.” Thanks to iterative note taking, I arrived at the point where I was ready to write it. Because I had so many notes, I had a very clear idea what it should be, and I was able to write it very quickly. Not that this has to be writing a whole programming language. It could just be making better coding choices on the next project.
“You are literally super-human.”
Remember that you are a programmer and the world is information. You’re wielding the most powerful tool ever invented. Programming allows you to automate the world. You are literally super-human: a human plus a computer. It is incredibly empowering to realize you can automate away anything that impedes your raw, creative power.
Beyond the Code
Let’s take a step back and address lifestyle and health. You are using your brain and body, so maximizing both matters. Take care of yourself. Pay attention to your nutrition, exercise, physical, emotional and social health.
If you don’t know where to start, research mindfulness practices. I highly recommend yoga. It’s both mindfulness and the perfect antidote to sitting in the chair too much. But it isn’t for everyone. Find what works for you. Most mindfulness practices start with “mind-quieting” – giving your mind a break from thinking about everything so that you can come back more productive to work or your personal life. Even just taking regular walks can have a huge impact. There is a lot of research behind meditation that validates the effectiveness of how taking a little bit of time every day can make you more effective over the rest of your day and life.
Get inspired. When you are programming, your best productivity comes from being in “flow,” and that requires inspiration. I strongly believe in the power of outside stimulation to inspire: music, smell, texture, and taste… Inspiration doesn’t always come from being in the office and staring at a wall. Of course, everybody’s different and so is each day. Every 24 hours is an opportunity to find more of the mix that works for you. That’s why it’s an iterative process. Reflect on yourself, make improvements in small ways, and perhaps most important, have faith in those little improvements. They operate like compound interest. Even a 1% interest rate, when compounded, can offer an enormous return over time.
Productivity and Mastery
Productivity is about mastery, incremental improvements, an ever-self-improving process of delivering more value with less time. To deliver more value, we seek to understand the essential problem and its essential solution. This is the theoretical, maximally best solution to a problem. It’s not an attainable goal; it’s an ongoing practice. We must strive to understand the customer’s problem deeper than they understand it themselves, and we must strive to master our medium and produce ever-better solutions to those problems.
As we strive to do more, we are also striving to do it with less. Doing less is about eliminating wasted pieces of time: all the little distractions throughout the day. Some distractions are good. They might submerge you into moments of reflection or inspiration that act as nutrients for productivity. Yet many distractions simply break our flow. Determining which distractions are wasteful and those that are beneficial comes with self-awareness practices and, as with anything of value, remains an iterative journey.