Designing strong public digital institutions

Society needs effective public organisations, and no organisation can be called effective unless it can also be called digitally effective. Yet as the National Audit Office put it, in “twenty-five years of government strategies and countless attempts to deliver digital business change, there is a consistent pattern of underperformance.”

A question I often talk about with civil servants is: how ought public digital organisations be arranged? If a department is directly responsible for delivering digital services, what is the right way of organising its people, and its executive team, to deliver the most benefit?

What do we want?

We should start with what we want public digital organisations to do. You can debate the finer points forever, but I would suggest the following broad brush statements as things most of us would sign up to:

1. Deliver value for users. Specifically, deliver products & services that are valuable, usable, feasible and viable.
2. Run at reasonable cost. Especially in relation to benchmarks set by other organisations, public and private.
3. Be transparent. Public works should be equally (if not more) accountable for their performance than private ones.
4. (For governments) Do what the public wants. In particular, by making good on the commitments of their elected representatives.

Considered together, these aims aren’t so different to those of private sector organisations. What it’s fundamentally about, as Michael Porter would say, is optimising return on invested capital – albeit in terms of social good rather than profit.

Creating products customers love

The purpose of public digital organisation design, then, is to create an organisation that accomplishes those four goals: one that delivers value efficiently, transparently and democratically. For the purposes of this post, I’m going to focus just on the first two: value and cost.

To organise for delivering value at reasonable cost, we need to understand how successful digital products and services are brought about. There are four attributes of every successful digital product:

1. Valuable. It’s useful to its users.
2. Usable. Users can find and use it.
3. Feasible. It can be built at reasonable cost.
4. Viable. It’s ethical and in line with our values.

There is far more that could be said about the process for creating products that meet these four criteria, but this isn’t the place for that. For those who want more on the modern product development process, Marty Cagan’s Inspired: How to Create Tech Products Customers Love is the place to start.

For our organisational design purposes, the key takeaways from the literature on product development best practice are:

• Successful digital products are almost always built by long-lived multi-disciplinary teams that have broad power to shape their own work. We call these empowered product teams.
• Most product failures come from incorrect assumptions made in specifying the product. Specification of products has become its own specialised process, called product discovery.
• The most effective teams mix continuous discovery with continuous delivery. That is, they understand the problem and implement the solution in parallel. They also continue to look after the product long after the initial release.

For our organisation design to be successful, then, it must above all support empowered product teams. Our ultimate success as an organisation will consist almost entirely in the success (or lack of success) of our empowered product teams. It makes sense to ask then, what do teams need?

The needs of teams

Of course, product teams need many things to be successful, and it would be easy here to enter a book-length description of the culture and ways of working of successful organisations. To keep things simple, we are just going to focus on the really fundamental ones. The absence of any of these will almost certainly make the investment in the team a write-off:

1. The need for strategy. By definition, a successful team has succeeded by some measure. Clear goals, and a clear articulation of the team’s role in the value chain, are vital for empowered product teams to succeed.
2. The need to define their own work. Most product teams exist inside hierarchies, which introduces a risk that tasks will be “handed down” fully-formed. But where teams put their discovery effort needs to be shaped by goal-setting, not task-setting.
3. The need for skills. Product discovery and product delivery are expert processes that require specialist skills that are hard to learn and hard to hire in. You wouldn’t get a taxi from a driver without a license, and you shouldn’t assume anyone without training can work on a product team (even in the supposedly “less technical” roles).
4. The need for performance monitoring. The best teams operate inside a feedback loop where they are strongly rewarded for accomplishing progress in meeting the goals outlined by the strategy. This is very different to merely checking whether a delivery deadline was hit.

So, now we have two ways of testing our organisation design. Does our design increase the likelihood that successful digital products will emerge, via product discovery and product delivery conducted by empowered product teams? And is it likely to provide those teams with their essential needs?

A C-suite for empowered teams

Let’s look at how our C-suite might support empowered product teams directly:

• The need for strategy: provided by a Chief Product Officer/VP Product. Organisation strategy, i.e. that owned by the overall leader of the organisation, is necessary here but not sufficient. Translating organisation strategy into something multiple product teams can efficiency execute on is one of the functions of product management.
• The need to define their own work: provided by the Chief Technology Officer and CPO (and sometimes Chief Delivery Officer), who are the senior flagbearers for the idea that teams define their own work. Beware general managers with “vertical responsibility,” they are the most incentivized to leverage the hierarchy to interrupt the normal running of teams. (If you find yourself needing general managers in product development organisations <1000 people, you may have weak product and engineering managers.)
• The need for skills: provided by the CTO, CPO and VP Talent/HR. The existence of these roles is itself a green light to talent. Many engineers I know will simply not consider joining an organisation without a CTO who at some point in their career wrote real production code. And elevating what is sometimes dismissed as “recruitment” to the top table reflects its existential importance to product development organisations.
• The need for performance monitoring: provided by the product and data functions, under the sponsorship of the CPO and/or a dedicated Chief Data Officer.

Of course, one could debate the niceties of such an arrangement forever (and everyone does). The point is: an executive team designed consciously to support empowered product teams is more likely to be successful than one that is designed around other goals.

Functionally ubiquitous

The functional model, where the organisation is led by functional leaders (product, technology, etc.), and most individuals report both to a function and a product team, is ubiquitous in the private sector. It’s the organising basis of Google, Apple, Amazon, Netflix and Spotify to name a few. It’s so ubiquitous in fact that it’s hard to find examples of anyone doing anything else. The logic of it is compelling: empowered product teams take the vertical responsibility to deliver value for customers. Functions take the horizontal responsibility to build organisational capability.

This begs the question: why is it still so uncommon in the public sector? Why are such organisations still regularly organised into “directorates,” “programmes” or “units,” and why are there so few functional leader in senior positions?

It’s true that even in tech companies “divisions” of various kinds are common. Google for example has seperate SVPs responsible for YouTube and Search. But these are there to organise many tens or even hundreds of product teams, executing on strategies that may be different or even opposite. They are more analogous to the high level divisions of government (health, transport, justice and so on) than to individual civil service programmes or directorates.

Noble integrations

One reason the functional model struggles to take root is that many public services are major operations already, and leaders seek to integrate the “digital component” of the service into the broader operation. This is a noble aim, and seems to fit the logic of providing users a joined-up experience across different touch points.

The inconvenient truth though is that product development simply cannot be run in the same way as a back office, policy team or a customer service center, and leaders rarely have the breadth of expertise to span the gap. The talent is also exceptionally hard to attract in this context, since technologists know they will end up working for people who don’t understand what they do. That’s why these attempts at well-intentioned “noble integrations” of product development into existing operations almost always end in outsourcing.

If you are in this situation, setting up a product development organisation under its own management – with a clear interface with existing service operations – is much more likely to be successful than trying to nest product development inside existing management structures.

The single wringable neck

Another reason government agencies resist the functional organisation model is their cultural preference for accountability resting on one person, usually via a designated “Senior Responsible Owner.” It’s a compelling idea. After all, we’ve already set out accountability as a key objective of our organisation design. Similar logic supported the introduction of early product managers. If we are all focused on making our bit of the value chain hum, who is focused on making sure the whole chain actually delivers for our users?

Advocates of the functional model have a standard answer to this: of course there is accountability, they say, teams are accountable. But this is unsatisfying for people accustomed to a single wringable neck. Sure, we have the concept of “product trios,” where teams are led by a product manager, a technology lead and a designer (or in some versions, a delivery manager). And we know we can apply the same principle to groups of teams. But this is still three people to talk to, not one!

I have a harder answer for you: you need to choose. You can have empowered product teams – alongside single accountability for functions – or you can have single accountability for delivering something. Unless your product development organisation is large (thousands of people), you will tie yourself in knots trying to have both.

As we have seen, empowered product teams need the skills that will only be acquired and nurtured by an organisation that represents those skills at the top table. They need healthy, vibrant functional strata that continually improve the performance of every discipline. They need a particular form of goal-setting that will only be provided by product strategy – it cannot come from a delivery roadmap. They need to work in a healthy engineering ecosystem that accelerates rather than hinders their performance, driven by a clear and long-lasting technology strategy. They need to define their own work, not be subject to the whims of hierarchy. And their performance has to be measured against the organisation’s priorities, not in terms of whether or not they make their boss happy.

Single delivery accountability in a product development organisation cannot rest with anyone other than the head of that organisation. That could be someone who leads all product development across the piece (for example, Condé Nast have a Chief Product & Technology Officer, the FT have a Chief Product and Information Officer), or it could be the CEO.

Enabling teams

Another key strand of the case for functional organisations is that, beyond just catering for the basic needs of empowered product teams, they actually actively accelerate them. How?

By centralising and optimising processes common to all teams. For instance, all product teams have various common engineering needs, like ways of managing code, setting up infrastructure and running releases. Once this is provided as a shared service, teams can forget about it.

Another example is user research operations. Every product team typically needs to recruit participants for user research, and maintain an archive of all the research they’ve conducted, but no one relishes how much time all this takes. By centralising the process we can streamline it, and give teams back time to focus on getting stuff done.

In an organisation split into vertical fiefdoms, these capabilities tend to be underdeveloped. Their absence turns up as a hidden cost on the balance sheet of each project. The investments only make sense if the product development system is seen holistically. Once it is, the potential to go enormously faster is huge.

Wrapping up

It’s telling that some organisations vest so much importance in single accountability for delivering outcomes. At its heart, this belief tries to skirt the complexity of how successful digital products are actually created. The idea is there is one person you can turn to who can be held to account through generic peformance management techniques, without the leader needing to understand the nitty-gritty of how their world works. From a certain angle it even follows the logic of empowerment.

The truth is we need to ask for more from the leaders of our public digital organisations. We need them to understand how the organisation should function (by, for example, spending a few years working on a product team), such that they can step in to correct it. Or we need them to appoint someone to run product development who does.

The orthodox wisdom of “integrating digital into everything we do” sounds great, but actually undermines the status of product development as a highly specialised expert process. A process performed by some of the highest paid and most in-demand talent in the market.

The public sector’s failure to understand how to organise itself is obvious in the huge size of the private agency sector that serves government. These companies hire brilliant, public-minded technologists and put them in a position where they can work on the fascinating problems government has to offer, without having to contend with its organisation structure. They use their profits, paid for by taxpayers, to fund internal talent development programmes that build their capabilities even further. Private agencies get stronger, public bodies get weaker, and value for money for the taxpayer diminishes. Eventually, even the people who could write a sensible procurement contract jump ship, and public agencies become effectively captured.

It does not have to be this way. The generation of technologists coming through now ask one question before every other: “how will my work make the world a better place?” Every private sector organisation I talk to is desperately scrabbling to articulate its business model in the terms of social purpose.

If public organisations can organise to empower these people, it will inevitably attract them. And from there, extraordinary things are possible.