Cantlin's blog

Designing strong public digital institutions

3140 words from 20 September 2021
"We fool ourselves if we set up a false dichotomy between markets and governments, since the functioning of one depends absolutely on the other. Whether government is big or small is a political choice that countries can make but, whatever your politics, the effectiveness (or lack of it) of government is immensely important."
– Sir Michael Barber
"Current and future public services are dominated by digital change."
– National Audit Office

Society needs effective public organisations, and no organisation can be called effective unless it can also be called digitally effective. Yet as the National Audit Office put it, in “twenty-five years of government strategies and countless attempts to deliver digital business change, there is a consistent pattern of underperformance.”

A question I often talk about with civil servants is: how ought public digital organisations be arranged? If a department is directly responsible for delivering digital services, what is the right way of organising its people, and its executive team, to deliver the most benefit?

What do we want?

We should start with what we want public digital organisations to do. You can debate the finer points forever, but I would suggest the following broad brush statements as things most of us would sign up to:

  1. Deliver value for users. Specifically, deliver products & services that are valuable, usable, feasible and viable.
  2. Run at reasonable cost. Especially in relation to benchmarks set by other organisations, public and private.
  3. Be transparent. Public works should be equally (if not more) accountable for their performance than private ones.
  4. (For governments) Do what the public wants. In particular, by making good on the commitments of their elected representatives.

Considered together, these aims aren’t so different to those of private sector organisations. What it’s fundamentally about, as Michael Porter would say, is optimising return on invested capital – albeit in terms of social good rather than profit.

Creating products customers love

The purpose of public digital organisation design, then, is to create an organisation that accomplishes those four goals: one that delivers value efficiently, transparently and democratically. For the purposes of this post, I’m going to focus just on the first two: value and cost.

"Strong tech-product companies know they need to ensure consistent product innovation. This means constantly creating new value for their customers and for their business. Not just tweaking and optimizing existing products (referred to as value capture) but rather, developing each product to reach its full potential."
Marty Cagan

To organise for delivering value at reasonable cost, we need to understand how successful digital products and services are brought about. There are four attributes of every successful digital product:

  1. Valuable. It’s useful to its users.
  2. Usable. Users can find and use it.
  3. Feasible. It can be built at reasonable cost.
  4. Viable. It’s ethical and in line with our values.

There is far more that could be said about the process for creating products that meet these four criteria, but this isn’t the place for that. For those who want more on the modern product development process, Marty Cagan’s Inspired: How to Create Tech Products Customers Love is the place to start.

For our organisational design purposes, the key takeaways from the literature on product development best practice are:

For our organisation design to be successful, then, it must above all support empowered product teams. Our ultimate success as an organisation will consist almost entirely in the success (or lack of success) of our empowered product teams. It makes sense to ask then, what do teams need?

The needs of teams

"Amazon relies heavily on autonomous, single-threaded teams... these teams keep the company nimble, moving quickly with a minimum of external friction, but their autonomy must be paired with precise goal-setting to align each team's independent plans with the company's overarching goals."
Colin Bryar and Bill Carr

Of course, product teams need many things to be successful, and it would be easy here to enter a book-length description of the culture and ways of working of successful organisations. To keep things simple, we are just going to focus on the really fundamental ones. The absence of any of these will almost certainly make the investment in the team a write-off:

  1. The need for strategy. By definition, a successful team has succeeded by some measure. Clear goals, and a clear articulation of the team’s role in the value chain, are vital for empowered product teams to succeed.
  2. The need to define their own work. Most product teams exist inside hierarchies, which introduces a risk that tasks will be “handed down” fully-formed. But where teams put their discovery effort needs to be shaped by goal-setting, not task-setting.
  3. The need for skills. Product discovery and product delivery are expert processes that require specialist skills that are hard to learn and hard to hire in. You wouldn’t get a taxi from a driver without a license, and you shouldn’t assume anyone without training can work on a product team (even in the supposedly “less technical” roles).
  4. The need for performance monitoring. The best teams operate inside a feedback loop where they are strongly rewarded for accomplishing progress in meeting the goals outlined by the strategy. This is very different to merely checking whether a delivery deadline was hit.

So, now we have two ways of testing our organisation design. Does our design increase the likelihood that successful digital products will emerge, via product discovery and product delivery conducted by empowered product teams? And is it likely to provide those teams with their essential needs?

A C-suite for empowered teams

"One will weave the canvas; another will fell a tree by the light of his ax. Yet another will forge nails, and there will be others who observe the stars to learn how to navigate. And yet all will be as one. Building a boat isn’t about weaving canvas, forging nails, or reading the sky. It is about giving a shared taste for the sea..."
Antoine de Saint-Exupéry

Let’s look at how our C-suite might support empowered product teams directly:

Of course, one could debate the niceties of such an arrangement forever (and everyone does). The point is: an executive team designed consciously to support empowered product teams is more likely to be successful than one that is designed around other goals.

Functionally ubiquitous

"It was a strong functional model for digital and technology that got Britain to its world leading position; and it is the degradation and depletion of that functional model that explains much of the backsliding since 2015."
Lord Francis Maude

The functional model, where the organisation is led by functional leaders (product, technology, etc.), and most individuals report both to a function and a product team, is ubiquitous in the private sector. It’s the organising basis of Google, Apple, Amazon, Netflix and Spotify to name a few. It’s so ubiquitous in fact that it’s hard to find examples of anyone doing anything else. The logic of it is compelling: empowered product teams take the vertical responsibility to deliver value for customers. Functions take the horizontal responsibility to build organisational capability.

This begs the question: why is it still so uncommon in the public sector? Why are such organisations still regularly organised into “directorates,” “programmes” or “units,” and why are there so few functional leader in senior positions?

It’s true that even in tech companies “divisions” of various kinds are common. Google for example has seperate SVPs responsible for YouTube and Search. But these are there to organise many tens or even hundreds of product teams, executing on strategies that may be different or even opposite. They are more analogous to the high level divisions of government (health, transport, justice and so on) than to individual civil service programmes or directorates.

Noble integrations

One reason the functional model struggles to take root is that many public services are major operations already, and leaders seek to integrate the “digital component” of the service into the broader operation. This is a noble aim, and seems to fit the logic of providing users a joined-up experience across different touch points.

The inconvenient truth though is that product development simply cannot be run in the same way as a back office, policy team or a customer service center, and leaders rarely have the breadth of expertise to span the gap. The talent is also exceptionally hard to attract in this context, since technologists know they will end up working for people who don’t understand what they do. That’s why these attempts at well-intentioned “noble integrations” of product development into existing operations almost always end in outsourcing.

If you are in this situation, setting up a product development organisation under its own management – with a clear interface with existing service operations – is much more likely to be successful than trying to nest product development inside existing management structures.

The single wringable neck

Another reason government agencies resist the functional organisation model is their cultural preference for accountability resting on one person, usually via a designated “Senior Responsible Owner.” It’s a compelling idea. After all, we’ve already set out accountability as a key objective of our organisation design. Similar logic supported the introduction of early product managers. If we are all focused on making our bit of the value chain hum, who is focused on making sure the whole chain actually delivers for our users?

Advocates of the functional model have a standard answer to this: of course there is accountability, they say, teams are accountable. But this is unsatisfying for people accustomed to a single wringable neck. Sure, we have the concept of “product trios,” where teams are led by a product manager, a technology lead and a designer (or in some versions, a delivery manager). And we know we can apply the same principle to groups of teams. But this is still three people to talk to, not one!

I have a harder answer for you: you need to choose. You can have empowered product teams – alongside single accountability for functions – or you can have single accountability for delivering something. Unless your product development organisation is large (thousands of people), you will tie yourself in knots trying to have both.

As we have seen, empowered product teams need the skills that will only be acquired and nurtured by an organisation that represents those skills at the top table. They need healthy, vibrant functional strata that continually improve the performance of every discipline. They need a particular form of goal-setting that will only be provided by product strategy – it cannot come from a delivery roadmap. They need to work in a healthy engineering ecosystem that accelerates rather than hinders their performance, driven by a clear and long-lasting technology strategy. They need to define their own work, not be subject to the whims of hierarchy. And their performance has to be measured against the organisation’s priorities, not in terms of whether or not they make their boss happy.

Single delivery accountability in a product development organisation cannot rest with anyone other than the head of that organisation. That could be someone who leads all product development across the piece (for example, Condé Nast have a Chief Product & Technology Officer, the FT have a Chief Product and Information Officer), or it could be the CEO.

Enabling teams

Another key strand of the case for functional organisations is that, beyond just catering for the basic needs of empowered product teams, they actually actively accelerate them. How?

By centralising and optimising processes common to all teams. For instance, all product teams have various common engineering needs, like ways of managing code, setting up infrastructure and running releases. Once this is provided as a shared service, teams can forget about it.

Another example is user research operations. Every product team typically needs to recruit participants for user research, and maintain an archive of all the research they’ve conducted, but no one relishes how much time all this takes. By centralising the process we can streamline it, and give teams back time to focus on getting stuff done.

In an organisation split into vertical fiefdoms, these capabilities tend to be underdeveloped. Their absence turns up as a hidden cost on the balance sheet of each project. The investments only make sense if the product development system is seen holistically. Once it is, the potential to go enormously faster is huge.

Wrapping up

"Believing that conventional management had stifled innovation, Jobs, in his first year returning as CEO, laid off the general managers of all the business units (in a single day), put the entire company under one P&L, and combined the disparate functional departments of the business units into one functional organization."
Harvard Business Review

It’s telling that some organisations vest so much importance in single accountability for delivering outcomes. At its heart, this belief tries to skirt the complexity of how successful digital products are actually created. The idea is there is one person you can turn to who can be held to account through generic peformance management techniques, without the leader needing to understand the nitty-gritty of how their world works. From a certain angle it even follows the logic of empowerment.

The truth is we need to ask for more from the leaders of our public digital organisations. We need them to understand how the organisation should function (by, for example, spending a few years working on a product team), such that they can step in to correct it. Or we need them to appoint someone to run product development who does.

The orthodox wisdom of “integrating digital into everything we do” sounds great, but actually undermines the status of product development as a highly specialised expert process. A process performed by some of the highest paid and most in-demand talent in the market.

The public sector’s failure to understand how to organise itself is obvious in the huge size of the private agency sector that serves government. These companies hire brilliant, public-minded technologists and put them in a position where they can work on the fascinating problems government has to offer, without having to contend with its organisation structure. They use their profits, paid for by taxpayers, to fund internal talent development programmes that build their capabilities even further. Private agencies get stronger, public bodies get weaker, and value for money for the taxpayer diminishes. Eventually, even the people who could write a sensible procurement contract jump ship, and public agencies become effectively captured.

It does not have to be this way. The generation of technologists coming through now ask one question before every other: “how will my work make the world a better place?” Every private sector organisation I talk to is desperately scrabbling to articulate its business model in the terms of social purpose.

If public organisations can organise to empower these people, it will inevitably attract them. And from there, extraordinary things are possible.