Metrics of Evolution - Dynamics and Metrics of Ecosystem Evolution - Platform Ecosystems: Aligning Architecture, Governance, and Strategy (2014)

Platform Ecosystems: Aligning Architecture, Governance, and Strategy (2014)

Part III. Dynamics and Metrics of Ecosystem Evolution

Introduction

image

Chapter 7. Metrics of Evolution

Abstract

This chapter describes the three roles of metrics of evolution: steering evolution, separating signals from noise, and helping manage tradeoffs. It then describes three overarching principles that guide the selection of metrics used to help guide evolution. We explore nine strategic and operational metrics of evolution spanning three time horizons: The short, medium, and long term. These metrics encompass resilience, scalability, and composability in the short term; stickiness, platform synergy, and plasticity in the medium term; and envelopment, durability, and mutation in the long term. We also explore how these metrics directly influence others in the longer term, and ones where improvements simultaneously bolster another metric. These metrics apply to both platforms and apps. For both platform owners and app developers, we also offer a few ways to tangibly measure each metric.

Keywords

resilience; scalability; composability; stickiness; plasticity; durability; mutation; business ecosystems

If you don’t know where you are going, any road will get you there.

Lewis Carroll

In This Chapter

• The roles of evolutionary metrics: steering evolution, separating signal from noise, and managing tradeoffs

• Overarching principles for metrics selection

• Differences between metrics of evolution and traditional software engineering metrics

• Short-term metrics: resilience, scalability, and composability

• Medium-term metrics: stickiness, platform synergy, and plasticity

• Long-term metrics: envelopment, durability, and mutation

The evolution of platform ecosystems is a journey, not a destination. In a journey, you need markers—pins on the map—to keep track of your bearings in order to decide whether you are indeed headed in the intended direction. Metrics of evolution are such markers.

Dictums in the popular press such as “platforms must innovate or die” are not very helpful in practice because it is often difficult to measure innovation. This chapter brings the lofty notion of platform innovation down to earth, with tangible metrics to measure different aspects of the amorphous notion of innovativeness over time. This chapter first describes the three roles of metrics of evolution: steering evolution, separating signals from noise, and managing tradeoffs. It then describes three overarching principles that guide the selection of metrics used to help guide evolution: an outside-in vantage point, focusing on the short term without losing sight of the long term, and the consideration of costs versus value of tracking any particular metric. We then explore nine strategic and operational metrics of evolution spanning three time horizons: the short, medium, and long term. These metrics encompass resilience, scalability, and composability in the short term; stickiness, platform synergy, and plasticity in the medium term; and envelopment, durability, and mutation in the long term. We also explore how these metrics directly influence others in the longer term, and ones where improvements simultaneously bolster another metric. These metrics apply to both platforms and apps; we therefore generically refer to these as “subsystems” in a platform’s ecosystem. For both platform owners and app developers, we also offer a few ways to tangibly measure each metric.

7.1 Three roles of evolutionary metrics

Metrics of evolution serve three roles in assessing the evolution of a platform or app: (1) they help steer evolution in ways that enhance its fitness in its evolving competitive environment; (2) they separate signals from noise to avoid pursuing dead ends and not miss out on promising opportunities; and (3) they help manage tradeoffs among design choices in forks along its evolutionary trajectory.

7.1.1 Metrics steer evolution

Passive adaptation to an unfavorable environment is the road to mortality. Evolutionary metrics focus attention on what’s important, not just on what is quantifiable. An obsession with translating a metric into a quantifiable measure can often result in measuring the wrong things. This can lead a platform owner and app developers to diligently pursue goals to produce the result measured by that metric—which is not what anyone in the ecosystem actually wants. MindSpring is an example of how wrong metrics can lead well-intentioned individuals to do the opposite of what is intended. In the late 1990s, MindSpring wanted to improve its customer service. So it started tracking the amount of time that its service representatives took to address a customer problem when they called its technical support hotline. It soon found that the duration of the calls significantly decreased, but, unexpectedly, customers became even more dissatisfied with its technical support. To its surprise, its technical support staff was rushing customers through support calls, sending them on wild goose chases to get them off the line, or outright hanging up on them. Mindspring’s support staff maximized precisely what was being measured, but producing exactly the opposite of the intended goal. Platform owners and app developers are therefore much better off with an approximate measure of a relevant evolution metric than with the precise measurement of a wrong metric. Just because a metric cannot readily be measured does not mean that it does not matter. Pretending that a property does not exist if it is hard to quantify can lead to faulty mental models and dysfunctional dynamics (Meadows, 2008, p. 176). Just remember that the singly critical long-term goal of a platform ecosystem is survival.

7.1.2 Metrics separate signal from noise

A second function of evolutionary metrics is to amplify meaningful signals and filter out noise. Tracking what’s important in a platform ecosystem is like looking for a needle in a haystack because of a low signal-to-noise ratio. Signal refers to things that really matter, or indicators of the evolutionary health of a platform. Noise is information other than the meaningful signal such as readily available data, factoids, assertions, and beliefs. Confounding signal for noise and noise for signal can result in strategic blunders by a platform owner and app developers that can lead them down blind alleys, push them to pursue dead ends, and even put them on a death spiral.

Software platforms and the rise of big data mean that platform ecosystems are inundated with previously unimaginable streams of real-time data. This overabundance of information creates a poverty of attention. Software developers have historically dealt with data points, not data streams. The existing professional toolkits of software metrics focus largely on short-term, operational metrics that are appropriate for projects with a clear end point and rely primarily on data points (see “Evolution Versus Metrics of Implementation Performance”).

Errors in judgment that result from misclassification of signals and noise are illustrated in Figure 7.1. Confusing signal for noise can result in a failure to make timely investments in an emerging technology and markets that subsequently turn out to be disruptive to the platform or an app (cell D in Figure 7.1). Confusing noise for signal can result in investing in emerging technologies and markets that turn out to be dead ends (cell B in Figure 7.1). Statisticians call these errors in judgment Type I errors (aggressively pursuing an opportunity that eventually turns out to be a mirage) and Type II errors (failing to pursue an opportunity that would eventually turn out to be big). The greater the technological and market uncertainty in a platform market, the greater is the likelihood of such errors in judgment. The Red Queen effect can make it challenging to recover from either one of these errors. The worst position to be in is the right half of Figure 7.1, where simultaneously committing both errors in judgment is guaranteed to put a platform on an accelerated death spiral. The metrics of platform evolution developed in this chapter alleviate the signal-to-noise ratio problem by reducing noise and amplifying weak signals. Good evolutionary metrics of platform evolution should ideally move you as close as possible to cell A in Figure 7.1. Real options thinking, described in Chapter 8, provides the framework for designing architectures that include strategic flexibility for coping with both technical and market unpredictability.

image

FIGURE 7.1 The evolutionary consequences of confounding signal and noise.

7.1.3 Metrics help manage tradeoffs

There’s an old saying in the software business that you can have a project on time, within budget, and of high quality—just pick any two. Evolving platform ecosystems involves similar tradeoffs, except their consequences are not immediately visible because they can span long timeframes. By focusing attention on multiple time spans, metrics help ecosystem participants recognize the long-term consequences of their immediate choices. In other words, they help deliberately and consciously manage tradeoffs that are otherwise invisible until much later.

7.2 Three guiding principles

Tracking multiple metrics can be costly and time-consuming, and not all metrics are worth tracking. Three overarching principles should guide a platform owner’s and app developers’ choice of metrics of evolution to track: (1) they should provide an outside-in vantage point, (2) they should focus on the short term but without losing sight of the long term, and (3) they should be worth more than their measurement cost.

7.2.1 Look from outside in

The first guiding principle for choosing metrics of evolution is to start from the outside, looking in. Responsiveness to existing and prospective customers' latent needs should overshadow everything else. The primary customers of a platform are app developers and the primary customers of apps are end-users. Starting on the outside focuses attention on what matters most in an evolving marketplace, potentially riddled with rival platforms and rival apps. An outside-in approach is a constant reminder that you, as an ecosystem participant, live and die with the ecosystem. The fate of a platform owner is inseparably intertwined with the fate of its app developers. It is particularly important for the platform owner to remember that competition exists among platform ecosystems, not among individual platforms. Nokia once dominated the mobile telephony business, but it was beaten not by Apple or Google but rather by their ecosystems. Nokia was not challenged by a firm, but by an ecosystem. One company, no matter how dominant or successful, is hardly a match for an army of companies rallying together as an ecosystem. It is chaotic, like a properly attired regiment of soldiers pitted against a much larger rag-tag band of revolutionaries. Eventually, the rag-tag band, in spite of its chaos, will likely prevail by its sheer overwhelmingness and diversity of self-organized tactics. Ecosystems—with a systems advantage and high collective evolvability—will usually overwhelm and leapfrog even the most leading firm. And in Red Queen competition, the platform that looks the most attractive to its app developers and end-users will win in the survival of the fittest.

7.2.2 Focus attention on the short term without losing sight of the long term

The second guiding principle for choosing metrics of evolution is to focus on the short term without losing sight of the long term. Measuring the evolution of a platform and its apps requires tracking the key properties that indicate whether it is evolving along a favorable or unfavorable trajectory. Evolution in complex systems occurs over long timeframes (de Weck et al., 2011, p. 65). Therefore, the tracked metrics of evolution must span shorter and longer time scales.

Let us momentarily go back to the idea of evolution as a journey. Trying to orchestrate a platform without good short-term metrics is like trying to drive a car without a speedometer. Trying to orchestrate it without medium-term metrics is like driving in an unfamiliar city without a map. Missing long-term metrics is like driving without a compass, particularly with your author’s sense of direction. As we subsequently explain in this chapter, what is short and long term is subjective and varies from one category of platforms to another. It should always be defined using the lifespan of comparable platforms as a reference point (Tiwana et al., 2010).

Another way to think about metrics along the three time horizons are as three lenses of a telescope; each provides a different view of what’s ahead. This is illustrated in Figure 7.2. Together they provide a much better view of reality along a platform ecosystem’s evolutionary trajectory.

image

FIGURE 7.2 Three lenses for measuring evolution in platform ecosystems.

Simultaneously using metrics for evolution over the short, medium, and long term provides three evolutionary advantages for platform owners and app developers:

Imagination-driven innovation. Using multiple lenses to track evolution provides a future-looking perspective, allowing ecosystem participants to see the platform’s technology not as it is today, but what it can become. Some of the most profound innovations of our era resulted from someone imagining a technology that was impossible then and then working diligently to make it a reality. Take the iPad, for example; the entire concept was envisioned in a 1988 inhouse video called the Knowledge Navigator that took over two decades to realize. (Google it.)

Recognizing the long-term consequences of today’s choices. Today’s design choices by platform owners and app developers can also have irreversible evolutionary consequences; seeing the connections requires a prepared mind using the linkages among short-, medium-, and long-term metrics of evolution. The lens that we fixate on influences not only what we do see but also what we do not see.

Avoiding force-fitting. Using just one type of lens will also subconsciously bias us to force-fit the unfamiliar with what we recognize. (Remember the story of the elephant and eight blind men in Chapter 5?) Tracking evolution metrics spanning multiple timeframes also mitigates the human tendency to overvalue short-term and undervalue long-term consequences of design decisions, which is pervasively observed in technology development decisions (Gamba and Fusari, 2009).

Using these three timeframes to track evolution in platform ecosystems provides us a telescopic view that prevents us from isolating the context from our decisions. However, this telescopic view also needs to be complemented by a “magnifying-glass view” that prevents us from missing the little operational details. This magnifying glass is real options thinking, described in Chapter 8.

7.2.3 The cost of tracking a metric should never exceed the value of tracking it

The third guiding principle for choosing metrics of evolution is that the cost of measurement should not be more than the value that you—the platform owner or app developer—get out of a metric. In practice, this means that app developers should focus more on app-specific metrics and less on platform-level metrics. However, app developers must also be cognizant that they will sink or swim with the platform unless they multihome. For platform owners, it is important to distinguish between public and private metrics. While private metrics are for the platform owner’s eyes, public metrics are the ones that app developers can use in their own work. These are typically app-specific metrics and ecosystem-wide metrics. To make sure that the right public metrics are used, stick with the overriding principle that a metric first should do no harm (i.e., not push the ecosystem in an undesirable evolutionary direction). The platform owner should also seek to automate data-gathering for assembling metrics where possible. This will lower app developers’ measurement costs and help them better self-organize around your platform in a more evidence-driven manner.

Evolution Versus Metrics of Implementation Performance

This chapter focuses on metrics of evolution but not implementation-oriented, technical performance metrics. Most software developers are familiar with software implementation performance metrics and there exist excellent reference books on that topic. Implementation performance metrics are necessary but insufficient preconditions to survive and prosper in platform markets. They focus heavily on projects with discrete end points or individual coders (“coder metrics”) as the unit of analyses but little on evolution, which is the focus of this book. In other words, they focus on how well a project was done rather than how well a system evolved to improve its fitness that predicates its survival and mortality. Put another way, they are more like a rearview mirror rather than a GPS screen.

Evolution should not be confused with maintenance (Kamel, 1987); the former is functionality-expanding and the latter is corrective in nature. The focus on software implementation metrics made perfect sense in the pre-platform paradigm, where efficiency and reliability—not evolvability—was the emphasis in developing software. These include defects density (number of defects per thousand lines of code), development productivity (development person-hours required per thousand lines of code), coder productivity (improvements in coder productivity over time) (Banker et al., 1992, 1994; Fenton and Neil, 1999). We therefore do not focus on software implementation metrics, which are backward-looking rather than forward-looking (Dekleva and Drehmer, 1997; Tiwana et al., 2010). We directly build on a few key concepts from the traditional software engineering metrics literature in developing our metrics of platform evolution. These concepts apply to the code used to implement both platforms and apps.

Release. A software release is a collection of compiled software code prepared for distribution and installation on end-user devices. It consists of replacement code implementing new features and functionality and modifying the undesirable characteristics of earlier releases. Quality of a release of a platform or app can be approximated by the number of patches released after its initial introduction to repair a defect in a small portion of the previously deployed code. These patches are often known as bug fixes and should not be confused with updates that add new capabilities to the system.

Version. Although releases are sometimes referred to as versions in colloquial use, we reserve the term version in this book to refer to a distinctly different concept: a variant or an edition that is a modification of a platform or app that adds or deducts features as the basis for price discrimination. A single release of a platform or app can have multiple versions at the same time (e.g., free vs. paid; or light vs. standard vs. professional).

Fidelity. Fidelity is the accuracy with which an app or a platform achieves the desired functionality (Messerschmitt and Szyperski, 2003, p. 56). A simplistic metric for fidelity is the inverse of defect density (defects reported per thousand lines of code), which can be compared across apps in the same platform (since they likely use the same programming language) but not across platforms. This simple metric captures real defects, which might be different from perceived defects, which are more noticeable and therefore matter more to end-users even when few real defects are present in an app or platform.

7.3 An overview of metrics of evolution in platform ecosystems

Metrics of evolution measure emergent properties of platform ecosystems. They tap into unfolding ecosystem behavior over time. Metrics of evolution described in this chapter therefore capture changes—increases or decreases—in some evolutionary property of platforms and apps over time rather than a snapshot at one point in time. Emergent properties—unlike lines of code, defect counts, system speed, and person-hours of development work—cannot be measured directly but only by their manifested effects and consequences. To reliably measure emergent properties, one must measure more than one manifestation of that property. We therefore use a bundle of three metrics for each of the short, medium, and long timeframes. Each of these metrics applies to both platforms and apps—any subsystem in a platform ecosystem. We therefore refer to these metrics as properties of a “subsystem,” where the subsystem can either be the platform or an app.

Each of these nine metrics can therefore apply to the platform or an individual app. These metrics can broadly be categorized as operational or strategic evolution metrics, and span the short, medium, or long term. Strategic metrics relate to the strength of the competitive positioning of a subsystem in its environment. Operational metrics are tactical measures that feed into strategic goals by breaking them down into workable actions.

The definition of short term and long term is inherently subjective and relative to the lifecycle of a population of comparable platforms. Each release of a PC operating system, for example, has a 5- to 10-year lifespan, mainframes about 40 years, and mobile phone operating systems about 2 years. It would be erroneous to define long term as the lifetime of one release of a platform. Instead, each release of the same platform should be added up to guesstimate what constitutes short term and long term. For example, the iOS platform in 2014 originated as iOS 1.0 in 2007, implying that the same platform has survived for 7 years. This realistically makes 7 years no more than the medium term in its lifespan. Similarly, Windows 8 in 2014 is simply a continuation of Windows 1.0 introduced in 1985, giving the Windows platform a lifespan of about 30 years so far. So a conservative definition of long term will span 7 and 30 years, respectively, for these software platforms. One could argue that iOS can be traced back to MacOS, which can be traced back to Unix; so iOS did not truly begin in 2007. This is where the concept of platform mutation, described in this chapter, makes the definition of the beginning of a platform more clearly delineable. A derivative platform’s age begins when it mutates—or splits off—into a different environment, free of any ongoing linkages to a parent platform.

The common underlying theme in the metrics of evolution is evolution speed, which is grounded in the evolvability of subsystems in a platform ecosystem. Evolvability is the capacity of a subsystem to efficiently change as new requirements, needs, and possibilities emerge (de Weck et al., 2011, p. 187). User needs and market dynamics evolve over time. As soon as a fertile new market niche is identified and a platform or an app emerges to successfully meet that need, it is likely to encourage copycats. It must evolve to remain differentiated. Architectural choices—a platform’s DNA—have an enduring influence on evolvability of a platform, the type of innovations that do and do not emerge around it, and the vibrancy of its ecosystem. However, whether its evolutionary potential is realized in ways that enhance its fitness in its competitive environment depends on how well its governance reinforces its architectural properties. This match between architectural properties and governance of platform ecosystems is what we callarchitecture–governance alignment, discussed in Chapters 10 and 11.

Here we focus only on metrics of evolution (Table 7.1), specifically sidestepping commonly used software implementation metrics. If they substitute for tracking evolutionary performance, fixation on software implementation metrics, used widely in software engineering practice, can be fatal in platform markets. We also do not cover metrics that can be used for implementing process control; existing metrics in the software industry—such as capability maturity metrics, integration test metrics, standards compliance, and build test metrics—can be used as a complement to the metrics of evolution.

Table 7.1

Definitions and the Strategic Versus Operational Nature of the Nine Metrics of Evolution in Platform Ecosystems

Image

Two types of relationships exist among the nine metrics of evolution: causal linkages (•) and correlations (Image). These are summarized in Table 7.2. Causal linkages, also indicated by the arrows in Figure 7.3, exist when one metric influences another in the longer term. Improving a causal metric is necessary to improve the other. Two metrics are correlated when actions that improve one metric also help improve the other, but one does not directly cause the other (Table 7.3).

Table 7.2

Relationships Among Metrics of Evolution

Image

•, one leads to the other; Image, the two are correlated, but not causally.

image

FIGURE 7.3 The nine metrics of evolution.

Table 7.3

Some Proxy Measures of Evolution in Platform Ecosystems

Image

Table 7.2 summarizes ideas to measure each of the metrics, as described in detail previously.

7.4 Short-term metrics of evolution

7.4.1 Resilience

Resilience refers to the degree to which a subsystem is able to maintain an acceptable level of service when it encounters an unexpected fault in another subsystem in the ecosystem or disruption in an external service with which it interoperates. Variants of the resilience property are fault tolerance, reparability, and robustness. This property describes the degree to which the subsystem is internally immune to uncontrollable external factors that are difficult for the subsystem’s developer to directly control (de Weck et al., 2011, p. 71). The property of resilience is an important short-term operational metric of evolution that complements common short-term metrics of performance such as reliability, stability, responsiveness, and performance.

Two key features of resilience are germane to subsystems in platform ecosystems vis-à-vis software systems from the pre-ecosystem paradigm. First, the subsystem’s internal immunity is to failure external to it, such as in another subsystem with which it interoperates or due to changes in its environment (de Weck et al., 2011, p. 90). Second, the important attribute is not failure avoidance but rather fast recovery to restore normal functioning—a capacity to bounce back—from a failure outside the subsystem. This means that it is robust to malfunctions in other subsystems in the ecosystem. This requires a design that can quickly isolate perturbations affecting the subsystem. At the platform level, this means that the platform is capable of bouncing back or recovering to resume functioning normally when an app malfunctions. At the app level, this means that an app is capable of functioning normally when either the platform or another app that it interacts with malfunctions or fails. Even in the best-designed platforms, apps can be resilient to perturbations from malfunctioning apps but are rarely resilient to failures at the platform level. Figure 7.4 compares subsystems with two levels of resilience.

image

FIGURE 7.4 Resilience.

7.4.2 Scalability

Scalability refers to the degree to which the functional and financial performance of a subsystem is size agnostic. This translates to the capacity of a subsystem to support a larger or smaller number of something—end-users, app developers, apps, or external software services. Scalability is a short-term operational metric of evolution that is applicable both to platforms and to apps. In practice, scalability then is the degree to which a subsystem can maintain its performance and function, and retain all its desired properties without a corresponding increase in its internal complexity (de Weck et al., 2011, p. 187). There are two important differences between scalability in platform ecosystems vis-à-vis traditional software systems. First, we normally think of scalability as the capacity of a software system to scale upward. However, in platform settings, the capacity to scale downward is just as important. Therefore scalability must capture the subsystem’s capacity to expand or contract, upward or downward (Parnas, 1979). Second, performance can mean technical performance in the commonly accepted view of scalability, but it can also mean financial performance.

If a subsystem lacks downward scalability, it might not be financially sustainable below a minimal threshold of end-users or apps. This often-overlooked problem has caused many a technically promising offering to crash and burn in platform environments. For example, Blackberry’s BBOS 10 platform lacked downward scalability because it was not sustainable below a minimal threshold count of apps on its new platform. That prevented the platform from successfully taking off. Similarly, apps that rely on network effects among end-users have been unsustainable because they did not reach a minimal critical mass of adopters for network effects to get started. An example of this is Apple’s Ping social network built into iTunes, which was subsequently shut down due to lack of downward scalability. As we subsequently describe, upward scalability is impacted most by the architecture of a subsystem and downward scalability by governance and business model decisions.

Scalability in terms of technical performance can be measured as the change in latency, responsiveness, error rates for each additional or fewer end-user or app at the platform level, and changes in counts of end-users or external services invoked at the app level.Figure 7.5 illustrates this for upward scalability of two subsystems. Scalability in terms of financial performance can be thought of in terms of where the breakeven occurs (i.e., where revenues exceed costs), illustrated in Figure 7.6. A less downward scalable subsystem will shift to the negative net revenue region faster as the scale of the relevant “something” (end-users or apps) shrinks. This means that the subsystem on curve B will be less financially sustainable than the subsystem on curve A as the something count decreases. The point at which the breakeven point occurs can be shifted through thoughtful governance and business model design, giving a subsystem greater downward scalability.

image

FIGURE 7.5 Technical scalability.

image

FIGURE 7.6 Financial scalability.

7.4.3 Composability

Composability refers to the ease with which internal changes can be made to a subsystem without compromising its integration with other interdependent subsystems, including those inside and outside the ecosystem. Composability can be measured in terms of effort, in person-hours, needed for reintegrating a subsystem with others in its ecosystem after one internal change is made to it. A subsystem that requires more person-hours of effort to integrate with the ecosystem is less composable than one that requires fewer person-hours of effort (see Figure 7.7). Other ways to think about this property is to think of the maintainability of a subsystem or its capacity to maintain interoperability after changes are made within the subsystem. The opposite of composability is brittleness, wherein a change inside a subsystem breaks its ability to interoperate with other subsystems within the platform's ecosystem with which it interacts. Although composability is a property of a subsystem, the same subsystem may be less composable in one ecosystem and more composable in a different one. Therefore, composability is a relative property definable only in the context of the ecosystem in which a subsystem functions. It is also a less immediately observable system property than responsiveness or speed.

image

FIGURE 7.7 Two levels of composability.

Composability is an important evolutionary property for three reasons:

1. Every dollar invested needs seven more for maintenance. Lifetime maintenance costs of software far exceed its initial development costs, by a margin of about 700%. We often tend to think of changes as corrective changes that fix flaws in the initial implementation. However, typically only about 25% of post-release maintenance expenses come from bug fixes and about 75% of maintenance expenses over the lifetimes of typical systems come from functionality-enhancing changes (Eick et al., 2001). Therefore, the costs of changing a subsystem can be more important over its lifespan than its initial development costs. However, consideration of maintenance costs is often an afterthought in most software projects (de Weck et al., 2011, p. 79).

2. Outside innovations are more absorbable. Composability influences the ease with which a subsystem within the platform ecosystem can evolve to exploit technological advances from outside the ecosystem to enhance its functional capabilities. An inability to change the software code within a subsystem quickly and reliably means that fleeting business opportunities are likely to be missed, putting it further and further behind in Red Queen competition. Well-designed subsystems should be able to rapidly absorb newness and incorporate innovations from the outside or else face obsolescence and competitive irrelevance.

3. Different parts of an ecosystem do not evolve in synchrony. Different subsystems within an ecosystem often evolve at different rates. These differential rates at which individual subsystems face the pressures and encounter the opportunities to change means that it is more useful to think of composability as ease of systems reintegration rather than systems integration. In traditional software systems, systems integration of modules or subsystems is often a one-shot activity. However, in platform ecosystems, it is an ongoing, dynamic activity where changes in any app or in the platform can introduce integration problems. It is therefore important to recognize the ongoing, dynamic nature of systems integration in platform environments. A single change in a single subsystem in a platform ecosystem can potentially require bringing all subsystems together again to realize overall ecosystem functionality. This means that the lower the composability of a subsystem, the less readily it can evolve independent of other subsystems in its ecosystem. This requires changes to be implemented in large blocks spanning multiple subsystems in an ecosystem rather than on a piecemeal basis as needed. On the other hand, changes can be made to composable subsystems at differential rates, allowing them to evolve independently at the pace that is individually appropriate for them. Composability is therefore a strategic metric of evolution.

Composability is a property of a platform as well as an app that interoperates with a platform. For a platform, this property means that a change does not require much effort to reintegrate it with apps within the platform’s ecosystem. It can also continue to work together with apps and with other platforms and external services that are not owned by the platform owner (de Weck et al., 2011, p. 93). At the app level, composability therefore means how readily a tweaked app can be reintegrated with the platform, and also possibly with other apps with which it might interoperate. Composability of a subsystem in the short term influences a strategic medium-term metric, its plasticity. Therefore, emergence of ecosystem-wide new capabilities is heavily influenced by the composability of subsystems within it. Interactions among them determine the ecosystem’s evolution speed. We subsequently describe how speed is one of the few reliable sources of competitive advantage that ecosystem participants can count on.

7.5 Medium-term metrics of evolution

7.5.1 Stickiness

Stickiness is a measure of “eyeball time” between a subsystem and its primary users. It taps into the subsystem’s ability to sustain their attention; attention is a scarce resource and the currency in platform markets. Stickiness addresses questions such as changes in how much a subsystem is used by its primary users after its adoption and how much sustained use it sees. Use of the subsystem can be active use (where the primary user directly interacts with a subsystem) or passive use (where the primary user uses the services of the subsystem in the background). The emphasis on active use is more meaningful in assessing stickiness.

Stickiness applies to both apps and platforms. The primary users of apps are end-users and the primary users of the platform are apps (rather than app developers). For apps, a good way to measure stickiness is the change over time in hours per session or the number of sessions per week by their end-users. Remember that we are interested in averaged changes—increases or decreases—in this metric to understand the evolutionary trajectory of the app. Figure 7.8 illustrates both increasing and decreasing stickiness of an app over time (e.g., over 3 months). Stickiness avoids the risk of overrelying on measuring its installed base—number of end-users who have purchased or acquired the app—to gauge its health (a common practice in the software industry). This might create an illusion that the app is doing well when in reality it might be seeing little use after its novelty has worn off for end-users. As apps are often subsidized by advertising revenues, ensuring stickiness is critically linked to the app’s business model as well.

image

FIGURE 7.8 App stickiness.

For a platform, its stickiness for its secondary users can be measured in the same way as it is for apps. However, for the primary users of a platform—app developers—its stickiness can be approximated as the change—positive or negative—in the API calls made by each app on average as the platform ages. A platform with an increase in API calls per app would indicate increasing platform stickiness for its app developers. Figure 7.9 illustrates both increasing and decreasing stickiness of a platform with an app as it ages.

image

FIGURE 7.9 Platform stickiness.

7.5.2 Platform synergy

The concept of platform synergy is derived from the broader concept of “synergistic specificity,” which is the degree to which a subsystem and another subsystem are made for each other (Schilling, 2000). This is a property of a subsystem that is always relative to something else, which is the referent. It is an attribute of a subsystem that is a combination of two concepts: synergy with another subsystem and specificity to that subsystem. It is useful to think of increases and decreases in synergistic specificity over time rather than its level at a snapshot in time. Our focus on platform ecosystems here means that we focus on synergistic specificity of an app relative to other apps for that platform rather than external services (e.g., an app designed specifically for an external service like Gmail or Dropbox). It is therefore most useful to think of synergistic specificity as a property of an app relative to a specific platform. Platform synergy in this narrower sense then refers to the degree to which an app increases—over subsequent releases—and the degree to which it uniquely exploits the evolving native capabilities of the platform in its own implementation. Put another way, it is the degree to which an app achieves greater functionality by being specifically designed for a particular platform. An increase in platform synergy means that an app is able to integrate and exploit the power of a specific platform to a greater degree over time. It also means, however, that the app becomes increasingly more dependent on that platform to deliver that functionality. An increase in platform synergy is therefore a double-edged sword, potentially increasing an app’s performance and integration with a platform on one hand and potentially increasing its lock-in with the platform on the other. This tradeoff is illustrated in Figure 7.10.

image

FIGURE 7.10 Tradeoffs in increasing platform synergy.

From a platform owner’s perspective, an increase in an app’s synergy with a platform can increase an app developer’s lock-in to the platform while also potentially enhancing its platform-specific performance. Lock-ins can either be coercive or value-driven, and platform synergy is a value-driven form of lock-in. Coercive lock-ins are those created by setup, learning costs, or complementary investments that app developers must make to develop apps for a platform, or through the use of proprietary standards. Such investments have limited salvage value in a rival platform. But such lock-ins are not sustainable in the long run and are technologically breakable using translators, middleware, emulators, and adapters. We describe these strategies in later chapters.

Value-driven lock-ins are primarily created by offering apps unique functional capabilities in the platform that differentiate an app created specifically for a platform vis-à-vis the same app created for rival platforms. Although this type of value-driven lock-in over app developers is technological capabilities-based, other forms of value-driven lock-ins include access to a larger pool of app adopters (market access-based), a stronger prospect for network effects from a larger platform user base, or knowledge-based (wherein greater access to end-user usage metrics data provides better actionable insights to app developers for developing their apps). Therefore, an increase in platform synergy noncoercively increases app developers’ costs for defecting to a rival platform. Given the potential for sustaining app developers’ commitment to the platform as well as increasing app performance, it is usually in the platform owner’s best interest to increase the platform synergy of apps developed for that platform.

From an app developer’s perspective, increasing platform synergy needs to be approached with more caution for two reasons. First, app developers must often make investments specific to a platform to increase platform synergy of their apps. These investments include acquiring technical skills of the programming language, APIs, and architecture of a particular platform that are more valuable in relation to that platform but of lesser value in a rival platform. This increases app developers’ exposure to the risk of opportunistic behavior by the platform owner (e.g., unilaterally changing the rules of the game at a later time). This risk is usually more pronounced for platforms that use proprietary interface standards than it is for ones that use open standards. Second, increasing platform synergy limits the degree of modularization of an app; increasing an app’s tie-in to a platform by definition gives it more monolithic properties in relation to the platform. The more platform-specific the functionality of an app is, the harder it is to change it without constraints. This can make it progressively more difficult to evolve the app independent of the platform. App developers can limit these risks in two ways. First, intentionally limit increasing platform synergy to avoid excessive lock-in with a platform, particularly in its early lifecycle stages. Second, multihoming—or simultaneously developing the app for multiple rival platforms—hedges against stranding risk (i.e., being stuck with a platform that eventually turns out to be the losing bet among rival platforms).

7.5.3 Plasticity

Plasticity is a pragmatic antidote to the alarming overuse of adjectives such as innovative and dynamic to describe subsystems in platform markets. Plasticity refers to the degree to which new releases of a subsystem delivers new functionality to its primary users that it was not originally designed to deliver. Plasticity is directly correlated with the rate of innovation that is accomplished in a subsystem. Malleability, extensibility, and the capacity to morph are other ways to envision plasticity of a subsystem. Plasticity is a direct measure of the emergent properties of software-based systems because it reflects a subsystem’s capacity to morph to meet new needs and possibilities that did not exist at the time of its original creation (de Weck et al., 2011, p. 85). Two nuances are important in assessing plasticity. First, the extension of capabilities should not have been preplanned. Preplanned capabilities reflect incremental refinements rather than plasticity. Second, the changes that result in plastic behavior should be internal to the subsystem. Plasticity therefore reflects a subsystem’s capacity to internally change to better fit changes in its environment (de Weck et al., 2011, p. 187), reflecting the ease and rapidity with which it can be evolved to meet the changing needs of its existing and prospective users or to exploit technological advances.

Primary users for the platform are app developers, and for an app are its end-users. It is important to remember that end-users are secondary—not primary—users of a platform. Plasticity of a platform is therefore the degree to which it can deliver new capabilities for the platform’s app developers. It can include gaining the ability to work with other platforms and external services that are neither owned by the platform owner nor originally designed to work with (de Weck et al., 2011, p. 93). For example, iOS does things today that were simply not possible at its inception. Plasticity of an app is the degree to which an app developer can rapidly, over subsequent releases, add new functionality, capabilities, incorporate user feedback, and address evolving user needs. A subsystem with high plasticity provides such new—related or unrelated—capabilities in addition to its originally intended functionality (de Weck et al., 2011, p. 86). Plasticity is comparable to properties akin to a Swiss army knife that can perform many different functions, including some that were not preplanned. The simplest metric for plasticity is a count of major features added to a subsystem averaged per major release over its lifetime.

7.6 Long-term metrics of evolution

7.6.1 Envelopment

Dominant platforms are often hard to displace because once they successfully create network effects, those network effects create large incentives to existing users to stick with them even if a technically superior alternative subsequently appears. Therefore, the functionality of the new solution has to be revolutionary enough to warrant switching to it plus foregoing all benefits of the existing network effects that are already in place. Envelopment is an alternative strategy to overcome this barrier for new entrants in platform markets.

We described envelopment in Chapter 2 as an evolutionary dynamic where one subsystem expands into the turf of another solution with which it has an overlapping user base by offering its functionality as part of an expanded multiproduct bundle of functionality (see Figure 7.11). Successful envelopment attacks have three requirements: (1) there must exist shared economies of scale across the original and the target solution markets, (2) the key leveraged asset should be relevant and readily transferable to the targeted enveloped market, and (3) the enveloped functionality must be adjacent to the enveloping subsystem’s domain. Envelopment depends on leveraging overlapping user bases and common components to offer a bundle that includes both the original subsystem’s functionality and the target solution’s functionality. The enveloping subsystem can either progressively tip-toe into the enveloped solutions’ space or outright swallow it. Adjacencies between enveloped solutions and enveloping subsystems can be of two key types:

1. Customer adjacency. The enveloping subsystem and the enveloped solution have substantially overlapping customer bases (Eisenmann et al., 2011; Evans and Schmalensee, 2007, p. 154). An example would be the existing pool of users of email applications that overlap extensively with the existing pool of digital camera users. Envelopment occurs here when the enveloping subsystem expands to offer the functionality of a different solution that is also used by a large percentage of its own users.

2. Value chain adjacency. The enveloping subsystem and the enveloped solution occupy different but proximate links in a value chain. Envelopment occurs here when the enveloping subsystem expands to span a different link in the value chain that was previously served by a different solution. An example would be digital photography software users versus digital printing services users or electronic scanner users. Similarly, the incorporation of inexpensive GPS chips in smartphones allowed them to progressively swallow the functionality of standalone GPS devices and mapping services.

image

FIGURE 7.11 An illustration of envelopment.

We call envelopment based on customer adjacency horizontal envelopment and that based on value chain adjacency vertical envelopment. As Figure 7.12 illustrates, vertical envelopment can be downstream or upstream.

image

FIGURE 7.12 Two types of envelopment in platform ecosystems.

Opportunities for envelopment arise primarily due to convergence of unrelated industries or applications either due to the convergence of previously disparate technology solutions or due to regulatory changes. Such envelopment opportunities arise from the application of existing or new technologies to deliver the functionality of the enveloped solution in a completely new way. For example, voiceover IP allowed software to envelop residential telephony; an increase in wireless data speeds and a simultaneous decline in costs allowed streaming services such as Pandora to offer the functionality of satellite radio in automobiles; Internet video-streaming services such as Netflix enveloped rent-on-demand entertainment services offered by cable television companies.

Envelopment becomes increasingly pronounced as product and service markets evolve into platform markets. This potentially poses a threat to nonplatform businesses that might find their bread-and-butter business under attack from a platform from an unrelated industry attempting to envelop their core market. Envelopment attacks often come from platforms in unrelated industries and from nontraditional, unexpected competitors, which makes incumbent solution providers especially vulnerable and unable to respond by matching the attacking platform or app’s functionality. For example, Craigslist versus local newspaper classifieds or Google Catalog versus print catalogs were not obvious substitutes, traditional competitors for the latter nor were they from the same industry. Successful envelopment attacks usually offer users of the enveloped solution a better value proposition based on different value-added functionality or a lower cost. For example, electronic catalogs such as Google Catalogs not only reduced mailing costs but also offered interactivity to clients such as L.L. Bean (which mails about 200 million printed catalogs each year). Google has progressively enveloped using its search platform electronic payment services (Google Checkout), software (Google Docs), email (Gmail), browsers (Chrome), advertising, and mobile OSes (Android) (Eisenmann et al., 2011). Similarly, iOS has progressively enveloped diverse solutions such as PDAs, hand-held gaming consoles, ebook readers, video recorders, cameras, and personal computers (Eisenmann et al., 2011).

Envelopment is a powerful way for new entrants in a platform market to overcome the competitive blocking caused by rivals’ existing network effects (Eisenmann et al., 2011). By using an envelopment strategy, a late entrant can become a dominant player in an adjacent market if it holds a dominant position in a related market. Recall the network effects shield platforms and apps from new competitors, who must offer functionality that is not merely better but so revolutionary that it is sufficient to convince end-users to abandon the network effects that they might already enjoy. This rarely happens in practice and the bar for defining revolutionary is often higher than new entrants presume. For example, Apple introduced FaceTime, which competed directly with Skype. However, the simpler interface, better integration with iOS devices, and lower bandwidth requirements were insufficient for it to displace Skype due to Skype’s strong network effects. In short, its technological merits were insufficient to entice Skype users to abandon their existing networks of peer users. Envelopment does not require revolutionary new functionality to displace a dominant solution (Eisenmann et al., 2011). Envelopment attacks instead harness network effects that previously sheltered the incumbent platform to their own advantage.

The key to breaking network effects barriers through envelopment is the notion of carryover (Adner, 2012). Carryover means that the enveloping subsystem simply replicates its existing network of users in the space of the enveloped solution. For example, iOS users of the iPhone provided an existing base of several million registered users to Apple when it introduced the iPad; the clever tactic was to use the same iOS account credentials across the two families of devices. Google has repeatedly used this tactic by adding enveloped services to its existing set of users, in essence instantly replicating their existing network of peer users in the space of the enveloped solution. Since the user bases of the Google platform and the target solution were substantially overlapping, the network effects that previously sheltered envelopment targets (such as Facebook) were no longer as protective. Such requirements for leveraging of a shared user base is the primary reason that envelopment attacks require a substantial overlap in user bases.

Envelopment is a significant evolution metric for both platform owners and app developers, who can both exploit envelopment opportunities and must also successfully rebuff envelopment threats in order to survive. The simplest way to measure envelopment is a raw count of successful envelopment moves by a subsystem and successful envelopment threats rebuffed. Another way of measuring successful envelopment moves is the percentage of new adopters of a subsystem that actively use the enveloped functionality after an envelopment move is completed.

For platform owners, envelopment therefore provides a way to grow the market occupied by a platform. It also offers a useful heuristic to anticipate envelopment attacks from adjacent platforms. For app developers, the envelopment threat not only comes from rival app developers but also from the platform itself. App developers must hedge against the risk that the platform owner might itself vertically integrate the functionality of an app into the platform, essentially wiping out instantaneously demand for the app. Although a platform owner might have its reasons for doing so (e.g., generating more competition in a critical piece of functionality or attempting to set a gold standard for app developers), this can damage the platform’s ability to attract app developers if this strategy is attempted too often. App developers can protect themselves from such attacks from platform owners by multihoming and embedding “real options” in the design of their app, as described in further detail in Chapter 11.

7.6.2 Durability

Durability refers to the competitive persistence of a subsystem’s advantages and distinctiveness over time (Barnett and Hansen, 1996; Pil and Cohen, 2006; Tiwana et al., 2010). Unlike performance durability that reflects a short-term property of resilience, competitive durability reflects a subsystem’s competitive endurance in a marketplace. An app that initially had 1 million adopters but only 50,000 actively using it 2 years later is therefore less durable than one that had half a million. Durability is therefore a more granular way of thinking of a subsystem’s survival and mortality over time.

Durability is an important long-term metric for how well a platform or an app adapted to improve its fitness in a changing competitive environment, rivals, and rebuffed envelopment threats. This can be measured objectively in two possible ways: (1) for an app or a platform, the change in the percentage of initial adopters who remain active users over an increasing time span (see Figure 7.13) and (2) for platforms, the change in the percentage of apps released that are subsequently updated at least once a year (see Figure 7.14). A coarser approach to measuring this at the ecosystem level is the change in the survival rate over time of a platform’s app developers. This metric can be compared across two platforms to compare the durability of rival platforms relative to each other.

image

FIGURE 7.13 App durability.

image

FIGURE 7.14 Platform durability.

7.6.3 Mutation

Mutation refers to the unanticipated, serendipitous creation of a spinoff platform or app that inherits some properties of the parent subsystem but with a completely different function than its parent (Tiwana et al., 2010). Unlike envelopment, which expands the subsystem’s functionality, mutation creates a distinct derivative system. Evolutionary biologists describe this process as speciation: taking an organism in one domain and transplanting it to another. When separated from its parent species and placed in a different application domain, the derivative system can develop very different characteristics from its parent species and grow quite distinct. In other words, derivative mutants diverge. Mutants can become distinctive enough to be the founders of the next generation of species of software systems; for example, the Mac OS platform mutated into the iOS platform in 2007. Mutation often results from the application of an existing solution to a new application domain. For example, face recognition technology combined with Amazon’s app resulted in Amazon’s Price Check app, which allows real-time price comparisons by its end-users in rivals’ stores. The radical discontinuity was not the technology itself but its application domain. The advantage of mutation is twofold: diversification and carryover. First, by creating a product or service that does not directly compete with existing ones, it offers a solution to being trapped in a Red Queen race by diversifying a subsystem’s core market (Meadows, 2008, p. 129). Second, the derivative system conserves some advantageous features of the parent subsystem. Evolutionary biologists refer to such genetic features that are highly conserved to mean that the same core features are redeployed in a very different species. Carryover that such inherited features facilitate refers to the process of leveraging the original subsystem’s ecosystem of users to jump-start the creation of a derivative, second ecosystem (Adner, 2012, p. 194). In software-based platforms, carrying over existing users of the parent subsystem to the mutated spinoff can give them a leg up in evolutionary competition. It exploits the original app or platform’s success in one setting to create an advantage in constructing a new ecosystem (Adner, 2012, p. 205). Carryover can therefore be used by platform owners to induce end-users as well as app developers to adopt the derivative platform. It can similarly be used by app developers to induce the original app’s end-users to adopt a derivative app. An example of such carryover is Google’s Google + social network: While Facebook and Twitter took about 2 years to reach a user base of 10 million users, Google + was able to do it in 2 weeks. Similarly, Apple used this strategy to introduce the iPad, putting it ahead of rivals even before it launched. Steve Jobs famously said when introducing the iPad to iPhone users that they already knew how to use the iPad.

Mutation applies to both platforms and apps in platform ecosystems, and can also apply to entire ecosystems. However, it is most often observed at the platform level. Measures of mutation in the evolution of a platform or app include the number of unrelated derivative platforms that grew out of a platform relative to rival platforms or the percentage of users at the outset of a derivative platform that were carried over from the parent platform. Apps can also mutate and grow into nested platforms—platforms within platforms.

7.7 Lessons learned

Evolution is a journey, and evolutionary metrics are the pin markers on the map. They help keep tabs on whether a platform or an app is headed in the intended direction.

Metrics of evolution apply equally to platforms and apps. Both are generically referred to as subsystems in this chapter.

Metrics of evolution play three roles. They help steer evolution to enhance a platform or app’s fitness in its evolving competitive environment; separate signals from noise to avoid pursuing dead ends and not miss out on promising opportunities; and help manage tradeoffs.

Three guiding principles guide the choice of a few metrics of evolution. Tracking multiple metrics can be costly. The chosen set of metrics must provide an outside-in vantage point, should focus on the short term but without losing sight of the long term, and they should deliver more value than what it costs to measure them.

Metrics of evolution measure changes—increases and decreases—in an evolutionary property over time. This makes them distinctively different from implementation performance metrics commonly used in software development, which are inherently backward-looking rather than forward-looking.

Metrics of evolution can be operational or strategic, and span the short, medium, and long term. The metrics of evolution encompass resilience, scalability, and composability in the short term; stickiness, platform synergy, and plasticity in the medium term; and envelopment, durability, and mutation in the long term.

Some metrics are causally linked and others are correlated. A metric in the shorter term can directly influence another metric in the longer term. Some others do not have such cause-and-effect linkages. Instead, improving one requires doing the same things that can improve another.

References

1. Adner R. The Wide Lens. New York: Portfolio; 2012.

2. Banker RD, Kauffman RJ, Kumar R. An empirical test of object-based output measurement metrics in a computer aided software engineering (CASE) environment. J Manag Inf Syst. 1992;8(3):127–150.

3. Banker RD, Kauffman RJ, Wright C, Zweig D. Automating output size and reuse metrics in a repository-based computer-aided software engineering (CASE) environment. IEEE Trans Softw Eng. 1994;20(3):169–187.

4. Barnett W, Hansen M. The Red Queen in organizational evolution. Strateg Manag J. 1996;17(1):139–157.

5. de Weck O, Roos D, Magee C. Engineering Systems. Cambridge, MA: MIT Press; 2011.

6. Dekleva S, Drehmer D. Measuring software engineering evolution: a Rasch calibration. Inf Syst Res. 1997;8(1):95–104.

7. Eick S, Graves T, Karr A, Marron J, Mockus A. Does code decay? Assessing evidence from change management data. IEEE Trans Softw Eng. 2001;27(1):1–12.

8. Eisenmann T, Parker G, Van Alstyne M. Platform envelopment. Strateg Manag J. 2011;32(12):1270–1285.

9. Evans D, Schmalensee R. Catalyst Code. Boston, MA: Harvard Press; 2007.

10. Fenton N, Neil M. Software metrics: successes, failures and new directions. J Syst Softw. 1999;47(3):149–157.

11. Gamba A, Fusari N. Valuing modularity as a real option. Manag Sci. 2009;55(11):1877–1896.

12. Kamel R. Effect of modularity on system evolution. IEEE Softw. 1987;(January):48–54.

13. Meadows D. Thinking in Systems. White River Junction, VT: Chelsea Green Publishing; 2008.

14. Messerschmitt D, Szyperski C. Software Ecosystem. Cambridge, MA: MIT Press; 2003.

15. Parnas D. Designing software for ease of extension and contraction. IEEE Trans Softw Eng. 1979;5(2):128–137.

16. Pil F, Cohen C. Modularity: implications for imitation, innovation, and sustained advantage. Acad Manag Rev. 2006;31(4):995–1011.

17. Schilling M. Toward a general modular systems theory and its application to interfirm product modularity. Acad Manag Rev. 2000;25(2):312–334.

18. Tiwana A, Konsynski B, Bush A. Platform evolution: coevolution of architecture, governance, and environmental dynamics. Inf Syst Res. 2010;21(4):675–687.


*“To view the full reference list for the book, click here or see page 283.”