Junk Change and Pastiche Product Management

There is a malaise affecting Change Management and Product Management 

Junk - Photo by Jessica Palomo on Unsplash

Junk [creativity] : Worthless writing, talk, or ideas

Pastiche : An artistic work in a style that imitates that of another work, artist, or period

I have a problem with the way that modern software development methodologies are being exercised by a significant proportion of the practitioners in the industry.

So much value has been placed on performing the mechanics of Agile or DevOps processes that the value of those processes is being lost under a sea of buzzwords and procedural nonsense.

This is not about "Is Agile Dead?" or "DevOps doesn't work!", I don't want to get involved with that sophistry. What I want to do is point out an increasing trend towards the belief that just doing Agile or DevOps will bring the benefits. The benefits stem from the good engineering practices that these methodologies promote, through a degree of formality, language and organisation. Without recognising and cherishing the engineering behind the methodology, too many teams are burning the trust of their investors and business managers.

So what is Junk Change [Management]?

• being unable to say why a change is required;
• making changes that have no associated metric;
• making changes that are mainly aesthetic;
• making changes that have no relation to end-user/Persona activity;
• refactoring without a reward;
• making changes that test/affirm no hypothesis about your users;
• using non-experts to approve changes;
• making non-testable changes.

And what is Pastiche Product Management?

Pastiche Development - Photo by Jesse Roberts on Unsplash

• having a product vision that never gets evolved or referenced;
• having Personas divorced from the product plan;
• Planning things that don't need planning;
• Estimating things that can't be estimated;
• Grooming a backlog with opinions rather than teeth;
• having "statement only" Stand-up meetings;
• having sprint tasks for BAU activities;
• doing retrospectives that change nothing;
• allowing unchallenged slow cadence/velocity;
• condoning rubber stamp PR code reviews and release approvals;
• writing unremarkable release notes;
• having no method for user feedback;
• ignoring "difficult" bugs.

Knowing "Why"

Knowing why you are making a change and why that change is important is the single most influential factor in promoting productivity and healthy product development. Ask yourself "why am I changing the colour of this button?" and then continue to ask why all the way back to the point where you reach the raison d'être for the business. If there is no realistic chain of connections then you don't have a valid reason for making that change.

The same goes for the mechanics of a methodology, knowing why you have stand-ups and why you do code reviews is very important for the success of those steps in the process. And everyone needs to know the reasons so they can hold the management team to account when a retrospective changes nothing or there are too many cards on the Kanban board.
Collectively knowing "why" is the basis of a shared purpose.

Understanding Users and Business Models

Large consumer goods companies such as P&G and Unilever spend a lot of money researching their consumers in many ways including H&P (habits and practices) studies, trying to understand them. Senior executives sometimes even visit the homes of tame consumers to try and gain a degree of empathy for their lives and see first hand how they use the product. The more successful technology companies also have a very strong customer focus, for example Jeff Bezos said:

"Focusing on what customers want or need has driven many of Amazon's most profitable business moves."

Linking all changes to a vision that chases down a series of consumers' needs and wants is key to good product management. They may not know they need it or want it yet, but being able to strategically address those needs at some point in the product roadmap is something that needs to be clear. A good example comes from looking at Google's dominance of the mapping space, analysis shows that the product decisions were made many years ago with some big investments as Justin O’Beirne's recent blog post unravelled; the whole roadmap is aimed at providing a superior user experience eventually (7 years later). Perhaps Google Maps is just a stepping stone in a greater product vision.
The team need to understand the user and their needs, there are a number of viable mechanisms for doing this but the one that I find works best for software solution is the Persona and macro level Use-case documentation. The team also need to be aware of the corporate mission and where that intersects the users lives, as well as a high level view of how the company plans to turn that point of intersection into a formal business model. A lot of developers and developer focused product managers fail to see the connection between good user modelling (Personas and Use-cases) and feature development. Without an accurate internal representation of how the users and customers the majority of product decisions are simply guesswork.
The user model defines the way in which you can ascribe value to features and hence prioritise, groom and make key decisions about the product and its roadmap.

Feedback, Metrics, Releases & Product Testing

"It's About Feedback and Change" is the title of an Agile Software Development paper by Williams and Cockburn (from 2003); it was written at a time when Agile was not yet the dominant methodology for software development, and while the acceptance of Agile has matured the key pillars of the agile manifesto seem to still be missing from the daily lives of too many of its practitioners. Feedback is a valuable element of the information required to make meaningful changes to both the software and the process of creating that software.
Direct user feedback is great, but it comes with a cost - either the cost of soliciting it or the cost of processing the un-moderated stream. Metrics provide a way of interpreting user behaviour and ascribing the value associated with those behaviours, either from changes in user retention, user engagement or more direct positive outcomes (up-sell or social vitality). Retrospectives and team meetings also have a cost and so productivity metrics are also a part of the development information required for a healthy agile team (when used positively).
Feature flags and A/B testing are common ways teams release change; and this works well from a process decoupling point of view, where the act of completing the development and the exposure to the end users are not tied to the same release step. But these tools are designed to allow the developers to finish and move on to the next task, and allow the product owners to work out the impact of the changes.

Hard Problems, Big Bad Bugs & Feature Leader Marketing

Commercial advantage comes from delivering products and services that either give your users a feature advantage, facilitate a cost reduction or provide any other significant differentiating factor (e.g. GDPR compliance). The source of these advantages for a tech company usually comes from solving a hard problem, for example the audio fingerprinting algorithm that Avery Li-Chun Wang created to launch Shazam. If you are not solving hard problems and constantly moving the dial in any significant way, and your competitors are, then you will rapidly lose your marketability and commercial leverage. Writing the paper is only step 1 of the product development, you need to get the functionality developed and put it in the hands of users, learn from how they use it and iterate to create the commercial gain.
Sometimes the hard problem is not visible to the user, perhaps it is security or issues of scale, perhaps it is a nasty bug that creates the operational problem to solve; I call these "nice problems to have" because you only get them if you are doing most things right, and so shying away from these problems is commercial suicide - you need to tackle them head on!
It doesn't matter if you are operating in a new or mature market (e.g. mobile phones) the only difference is that the more mature the market the more well known the critical features are, but you still need to be addressing both the known and the unknown/unexplored features; the product only needs to be one step ahead on each one to be market leading from a feature point of view. In other words, the product doesn't need to be years ahead in one area, it needs to be slightly ahead in all areas. Incremental improvement of features will only take you to the point where it starts to get hard, by focusing on the hard problems then you give your product that extra feature step at each iteration.

Branding and Aesthetics

There is a valid place for the importance of branding and aesthetic changes in product management. But the investment in those areas needs to be proportional to the value it brings. Branding can be very subjective and often a personal statement, but its value is found in the generation of familiarity, trust and making your brand synonymous with the qualities of the product you want to promote.

Summary

Think:

• Is the methodology actually working?
• Do the changes matter?
• Does everyone know why?
• What features make the difference?

Spike Notebooks - Better Agile Discovery for Developers

Spike Notebooks - Better Agile Discovery for Developers

tldr; apply data science tooling to software development research spikes.

Problems & Rabbit Holes

I don't like it when my team come to me with a problem and then don't offer up a solution. Sometimes they can see what is going wrong, occasionally they think they have a idea of how to solve it. Probably worst of all they have no idea how long solving the issue is going to take. Another common pain point is the phrase "yes I have looked at it and ..." which normally means they have just read a blog post like this one ;) and are walking blindly into someone else's solution.
In an agile process (during backlog refinement) this blind uncertainty is problematic and your typical agile team will attempt to resolve this through a spike, which is "a small task done to reduce uncertainty about a larger task". It is usually aimed at finding out why, what or how the next step should be completed.
The issue that consistently crops up is that the work done on the spike is valuable to the organisation as a whole, as it is a form of research, and what often happens is that some of the results of the spike will filter into a README.md, code comments or a gist or pastebin (if you are lucky) - but the majority of the technical effort is lost in a developer workspace or a defunct code branch. There is no real way to merge the spike work into the whole project reliably.

I have seen developers that will happily march into the thick of the problem and just start "fixing" things in the hope that it all comes our right and proper in the end. But I also see those same developers getting stuck in the rabbit hole: and by this I mean that for example, in order to solve problem (a) they introduced library (b) which caused a version incompatibility with (c) whose update requires a monkey patch (d) which meant we had to introduce CI/CD builds for component .... and so on down the rabbit hole. In other words, directly solving the problem means solving lots of problems as a chain.
At the other end of the scale a developer might read a blog or the documentation and establish that the solution is theoretically possible. This might be enough if the docs and the software are reliable, but this plan will still carry risk because it has not been proven at the critical points - and it is this risk that will blow any estimate of the resulting implementation work.
Solutions that require architectural changes are particularly prone to the kind of problems described above, either spending too long spiking the architecture or not de-risking the architectural change enough.

What is needed is a way of demonstrating that a solution exists, proving the risky parts of the solution and showing how they fit together. This might involve some coding, perhaps stepping into a rabbit hole, but then being able to backtrack on that and take a different path without having to revert a branch or comment out great swathes of code - all the time preserving the discoveries made and the thinking behind it. The goal needs to be to gather enough information about the solution to make a firm estimate of the work to implement with minimal uncertainly.

Welcome Data Scientists

Over the last 2 years I have been running a variety of transitional development teams, teams of data scientists and mixed data+engineer teams. The data scientists have brought in new skills, new approaches and previously unseen tools to the world of software development. Jupyter Notebooks are one such tool. A developer might think of a a notebook as a collection of markdown documentation, executable code snippets and recorded view-able results (including graphs and images); here is an example.

Photo by FloorTwelve on Unsplash

So can you use notebooks for spikes? They need to be:

• Easy for the team to use: file CRUD, correct language
• Replicate enough production code to work for a spike
• Review-able by other team members
• Kept with the code, but not overlapping it
• Link-able, documented and demonstrable
• Isolated enough to run alternate versions
• Lead into clear decision making (e.g. LADR)
• Seed TDD with fixtures and case outcomes
• Supporting teamwork

Are notebooks easy enough to use?
Notebooks used to be hard to run and quite a learning curve, but that is changing. The Jupyter Docker Stacks project is making ease of use a reality. And efforts like Jupyter Lab are all heading towards better drop-in work environments. The following is an example of how to create the basic python3 jupyter lab on a developer laptop using docker (because you are all using containers now right?):

docker run -d --name jupyter -v /home/${USER}/workspace :/home/jovyan/work -e GEN_CERT=yes -e JUPYTER_ENABLE_LAB=yes -p 8888:8888 jupyter/minimal-notebook

Then you can add (or compose) your own language kernel. There are many to choose from covering JavaScript, Java, Kotlin, TypeScript, Ruby, Go, C# and obviously Python. If you are using R or Python ML/data science libraries (e.g. tensorflow) then you can get them by using a more specialised stack image without having to go through an extra install step. Scripting languages are more aligned to this form of development, so there are more hoops to jump through to get a compiled language working.

Can you work on the spike?
This really depends on how well your software is written and structured. The principle is that you pull in the code you don't want to change, write the code you do want to change in the notebook and then execute the code and/or tests to see the differences. The notebook magic %load can be used to pull in your existing code or snippets of it. The %%script -bg magic allows you to start supporting systems (e.g. databases), and the %run magic will start your program from its entry point. If you need to temporarily overwrite code you can use %save to replace or patch a file.

How does the team review the spike?
I expected this to be a straightforward step, but it is more complicated than you think. Notebooks are stored in JSON format with a lot of timestamps and binary blocks and so a typical code pull review system will treat them as diff-able, which means you have to always click through to view the original. You can export the notebook to PDF or markdown and review that, but that takes you outside the main code process. Github renders the notebook in a readable form, but it is a bit flaky on larger notebooks. Bitbucket requires a plugin to work. The online renderers like https://nbviewer.jupyter.org/ will only show public notebooks. This leaves you with 2 choices: 1) run a private nbviewer, or 2) use a browser plugin. We opted for the plugin because of an IT policy, and I am planning on releasing the plugin to the chrome web store soon. By sticking to a process that closely follows code review the existing agile steps still apply. Incidentally there is an option to export the notebook as reveal.js slides.

Where does the notebook live?
We have ended up putting one notebook file (.ipynb) next to the code for each question we are asking, and then have a notebook at a higher level to link them and represent the spike as a whole. This takes advantage of the markdown documentation feature, which can create hyperlinks between relatively addressed files. So, for example, a high level notebook might be .../Spike - How do we make database X globally available.ipynb, which might refer to .../terraform/myproject/Can we tunnel Redis across regions.ipynb and .../src/mycomponent/dao/Can we encode X data in Redis.ipynb; then you might also have .../test/What is the failure rate of packets over tunnels.ipynb; and finally you might add later in the process .../test/What is the cost of using GCP spanner.ipynb. This accumulates knowledge and experience alongside the code it is affecting. Notice that spikes are not about answering all the questions, just the critical ones and the most unknown, engineers should be able to accurately estimate the work to complete at the end of the spike sprint. When we re-structure we move notebooks to an archive directory so that they don't get buried in the git history. We aim to create new notebooks rather than change notebooks because of the problems merging large blocks of high-change JSON.

How does this help with the development process?
In the example above we have an overall unknown (summary), an infrastructure unknown (temporary infrastructure), a code unknown (temporary code) and 2 operational unknowns (graphs and charts). At any point someone can jump back in and re-run the notebook and see if the result changes - for example if GCP Spanner pricing changes. In theory you can use notebooks for tracking A/B testing, but that is a whole different conversation.
Notebooks not only provide a neat way of hanging on to the knowledge created during the spike but also provide a means to quantify the benefits and risks of a change proposal (e.g. it will cost $900/m to expand to Australia maintaining 3-9s error rates). These kinds of information would normally get estimated and end up in someones drive as a spreadsheet instead of being openly referable in the code base. It is this transparency that provides the key benefit of this approach.
It is straightforward to reference a spike notebook if you are using decision tracking (e.g. LADR). You can use the %save magic or just use additional files to create data for TDD, especially if your spike is calling a real API with sample data making the fixture data as realistic as possible.

Who works on the notebook?
This really depends on the spike, but a common format is pairing a product manager/owner with a developer to flesh out the notebooks to start with and then let the developer loose from there. The nature of notebooks makes it hard to share them for co-editing, so physical (or virtual via screen-share) pairing works best. The pair will usually present their summary and selected highlights at a weekly show-and-tell.

Spike Notebooks

Spike Notebooks provide a clean and concise way to capture the researching of new software solutions. Handled well they can dramatically improve the transparency of the work done on the spike and lead to better sprint estimates, decision making and BDD/TDD approaches.