Gartner Report: Software 2020: Rearchitecting for the Digital World

Published: 28 January 2016

Analyst(s): 

Key Findings

  • New technologies are integral to emerging business strategies and are changing how IT leaders develop their architectures and organizations.
  • As technology becomes pervasive in all aspects of business, IT leaders are seeing more “IT work” being done outside of the IT organization.
  • Change is becoming so constant that long-term planning is impossible and our architectures, plans, and organizations need to adapt faster and more frequently than in the past.

Recommendations

  • Recast your technology strategy as part of the overall business strategy.
  • Break down the silos between applications, information and integration.
  • Enable everyone to apply new technologies to advance the enterprise.
  • Expect change, and build your organization and systems to adapt.
  • Remember that acquiring the technical skills is the easy part — changing the culture of the organization will be much harder.

Strategic Planning Assumptions

As of 2017, organizations that have fused digital and IT will outperform their peers in terms of reliability, customer experience and reputation.

By 2017, 50% of total IT spending will take place outside the formal IT organization.

Analysis

The Evolving Role of the CIO and IT

CIOs must ensure effective use of technology to achieve aggressive digital business needs and IT leaders must become trusted allies. Instead of being order-takers, CIOs should be on an equal level with other senior decision makers, becoming the strategic voice in the team on the use of technology and information.

Too many CIOs wait to be informed of requirements instead of directly influencing the future of the business. Change is happening too fast for that passivity to continue. Building up CIOs as trusted allies will broaden their influence and empower effective decision making within their teams. This research explores the many dimensions of change that can be harnessed for digital business outcomes.

Facilitate Change

New business models based on emerging technologies are forcing business and IT leaders to change the way they acquire and implement all kinds of software. IT leaders need to start helping their enterprises change immediately. It is about more than just technology, architecture, or even the vendor landscape — a deep cultural change is needed that will require taking apart and rebuilding your strategies, systems and organizations. Traditional categories and specialties, such as applications, information management, and integration, are fusing together.

This research looks at how these trends are forcing IT organizations to change from several perspectives:

  • How do organizations develop strategies, architectures, and teams to apply these new technologies?
  • Who actually creates, delivers, and maintains solutions now that technology is becoming pervasive in all aspects of business, and change is having to be mastered with customers and ecosystems in mind.
  • Change itself is becoming so constant that long-term planning is impossible and, therefore, architectures, plans and organizations need to adapt faster and more frequently.

The Changing Nature of “How”

Digital business is driving new patterns of business activity in the market, which, in turn, is driving the changing nature of “how” IT solutions are delivered. These new patterns are less predictable than in the past, making it impossible to define and automate a fixed process and let it run long term. Systems need to be highly reconfigurable and organizations must similarly adapt. Strategies must be agile, with the ability to flex around unpredicted situations. Business moments demand improvised interactions.

Digital business is connecting people, businesses and things in both the physical and digital worlds. A sudden market opportunity may trigger a new order that a vast ecosystem of partners responds to, orchestrating its fulfillment, processing information on the order, gathering input from the Internet of Things (IoT) and taking actions to optimize the value chain and delight the customer. With more awareness of what is happening throughout the value chain, the fulfilment process can be more fluid, with different nodes making choices about how to best fulfill its part of the process.

The concept of a customer order itself is being transformed in the digital business: in addition to traditional products and services, most enterprises will offer digital services that add to the preceding value proposition, creating new business models that will evolve as we understand the customer’s moments of truth.

The fulfilment of an order is not just a physical delivery controlled by a single enterprise, but an orchestrated ecosystem of manufacturing, transportation, information providers and service providers. Participants in this ecosystem will rely on digital workplace tools to deal with the changing nature of their work and to engage their employees to collaborate on creating the ultimate customer experience.

To achieve success in this fluid environment, IT leaders need to change some fundamentals, such as developing a digital first strategy, ensuring iterative delivery of fused technologies, and designing architectures for serendipity.

A “Digital First” Strategy

In the old days, businesses could develop a business strategy and set some high-level initiatives that would be active for years. They filtered down to business units that would then execute that high-level strategy by creating products and services, and also to IT teams that would build internal systems to execute business processes and analyze the results. An annual planning cycle was good enough to make it work.

The technology needed today extends beyond the realm of traditional IT. IT needs closer coordination with operational technology to make and deliver products. Digital marketing software, data, and services are used to find and sell customers. And, of course, the IoT is making new services and new interactions possible. Technology is much broader than information technology and involves multiple parties and budgets.

Detailed planning is also becoming more difficult to carry out in this emergent ecosystem. The business strategy needs to guide the general direction of the business and set investment levels, but more detailed decision making will have to happen closer to the customer and be done much faster, rather than waiting for the next annual planning cycle.

This means that the technology strategy can’t simply be derived from the business strategy — it has to be part of the business strategy. All of the interrelated types of technologies needed for the digital enterprise need to be considered together in the context of the whole enterprise.

For a detailed look at mastering digital business, see the Gartner research collection “Embed Digital Business Into the Fabric of Your Organization.”

Analytics Everywhere

New application architectures are emerging that challenge traditional methods of planning and implementation. The traditional practice separated transactional and analytical systems. Applications generated data and it was dumped in a data warehouse for analysis — hopefully before it got stale. Complicated queries didn’t slow down order entry at the end of the quarter. Theoretically, insights gained from analyses would feed back and be used to improve the business processes in the transactional system.

Hybrid transaction/analytical processing (HTAP) is a major change in these practices. In-memory computing (IMC) is making it possible to complete transactions and analyses simultaneously and quickly on the same in-memory data store. It is now becoming possible to imagine software that can sense specific conditions, such as an impending supply shortage, as new orders come in, communicate them to a decision maker (who may or may not be human), analyze the situation to find alternate scenarios, and then take action to put the plan into effect. These HTAP applications require all the application, information, and integration tools at IT’s disposal to be used together in a single system.

To understand the foundations of HTAP, see “Hybrid Transaction/Analytical Processing Will Foster Opportunities for Dramatic Business Innovation.”

ITERATIVE DELIVERY

Building these types of software is tricky given the siloed nature of IT organizations. The CIO often has separate groups below him or her, such as:

  • Application teams who build transactional systems
  • Information teams who build analytical systems
  • Integration teams who wire it all together

The move to agile development and DevOps, combined with the new types of software systems needed, means that we have to break down the silos and start building teams with the necessary combined skills — application, information, integration, and business acumen — to deliver. Each team will be multiskilled and focused on delivering specific business capabilities.

Teams will be coordinated using enterprise-agile techniques to tackle large, complex digital business problems and cross-team architectural and user experience issues. Most importantly, these teams will approach the problem from the outside-in. They will be striving to reach a measurable business outcome but need to start with the customer and understand how they decide to buy (or use) the product or service.

An iterative approach will be used to understand what the customer needs before combining these technologies to generate prototypes to be tested and improved in rapid cycles.

For an essential overview, see “Ten Things the CIO Needs to Know About Agile Development.”

THE ROAD TO ALGORITHMIC BUSINESS

Ultimately, concepts like HTAP, combined with Web-scale computing and the IoT, can change the way business is conducted.

In the world of data, big data has changed how large volumes of fast-changing datasets are analyzed. Millions of open datasets are making available details about geography, infrastructure, demographics, weather and transportation to combine with our other sources. New analytical tools utilize this data to predict future events — and even to generate prescriptive advice on what to do next. Natural-language analysis is creating an even bigger source of information, letting us analyze what people are thinking based on their communications and interactions. Smart machines promise more automated analysis and diagnosis from these facts.

Processes are also changing quickly, with increased automation leaving people free to handle the more complicated, nonroutine processes. Case-based business process management helps to organize and automate workflows for these situations. The fast-changing data stream requires complex-event processing to detect situations requiring attention and notify the appropriate people, while also supporting automated, real-time decision making — where the organization’s policies allow for it.

Blending all these trends together, we are starting to see evidence of what we call “algorithmic business” — dynamically using the trust-based ecosystem to sense, and respond to, customer demands, while escalating new situations to associates.

See the following Gartner research for insight into the algorithmic business:

  • Special Report: “Advancing Business With Advanced Analytics”
  • “Digital Business Gives Rise to the New Economics of Connections”
  • “Architect Your Business to Sense, Respond and Create Business Moments”
  • “Predicts 2016: Advanced Analytics Are at the Beating Heart of Algorithmic Business”

Design for Serendipity

As you plan and build these architectures, remember how limited visibility is: the immutable decisions you make now could compromise your future. Step back and ensure that you are building options into the future — design for serendipity, rather than designing based on fixed requirements and existing architectures. This requires jettisoning many old and familiar methods and technologies.

Establish clear goals in the form of business outcomes and let the teams find their own way there. Old application architectures may stick around as systems of record, but new hybrid computing may be a mesh of service-oriented architecture (SOA), Web scale, and microservice architectures.

The days of the single vendor, on-premises software suite are gone; hybrid implementations (specifically cloud and on-premises) are becoming the norm. Traditional, monolithic applications are giving way to apps that are focused on completing specific tasks, using SOA to enable interoperation where needed. Enterprise data warehouses are giving way to logical data warehouses that assemble data from many sources on the fly and use IMC to analyze it and take action.

Finally, things are changing too fast for a command-and-control style of management to be effective. Governance will be based on leaders setting goals and guardrails, and enabling teams to make and execute detailed decisions in a distributed fashion.

For more details on design and serendipity, see “How to Apply Gartner’s Digital Humanism Manifesto.”

The Changing Nature of “Who”

As technology becomes pervasive in all aspects of business, IT leaders need to consider who will actually be performing tasks. Jobs and roles are changing — many of which were previously based on establishing truth and control, not dealing with uncertainty. New critical competencies are also emerging.

IT leaders need to shift the mindset from shadow IT to citizen innovation, embrace the consumerization of software, and respond to the changing vendor landscape.

Change Your Mindset: From “Shadow IT” to “Citizen Innovation”

The ability of lines of business to procure great IT resources is stronger than ever. Talented technical people are being placed into areas such as sales and marketing. For IT leaders, it’s becoming more important to build relationships with these teams so that they understand how best to support the essential outcomes of the business. An important aspect of relationship-building is a language shift from older, IT-centric terms and perceptions, to more inclusive business concepts (see Table 1).

Table 1.   Examples of Language Shift

From:

To:

“Shadow IT”

“Citizen Innovation”

Fight

Embrace

Unarchitected

Fit for purpose

Fragmented

Focused

Ungoverned

Guided

Not in budget

Budget expansion, restructuring

Source: Gartner (January 2016)

Technology investments outside of IT are often referred to as “shadow IT” — as though it is something to avoid, or even fight. But these technology investments are a reality now that digital business lines are becoming profit-and-loss centers and are demanding more direct control over their development resources. These IT resources embedded in the business have a depth of domain knowledge that can benefit the traditional IT organization. Embrace the pool of new resources and use new platforms and approaches to enable their work while enforcing standards and quality.

Distributed innovation is not a one-dimensional matter though. Getting the best out of citizen innovators requires a careful portfolio analysis of where their work is acceptable.

Figure 1. Shadow IT and Citizen Innovation

Research image courtesy of Gartner, Inc.

Source: Gartner (January 2016)

For more details about citizen IT dynamics, see “Embracing and Creating Value From Shadow IT” and “Citizen Development Success Depends on an Equal Partnership Between Business and IT Leaders.”

In this age of distributed innovation, new competencies are emerging and evolving from traditional roles. Specifically, people that traditionally had a tight domain of control have to readjust to unknown outcomes. Enterprise architects become storytellers and influencers; business architects become anthropologists; integrators and taxonomists work more in real time, like jazz musicians, responding to unanticipated changes as they happen; project and product managers serve as conductors, creatively connecting the various pieces to achieve results. In this era, control gives way to influence and planning cedes to improvisation.

Table 2.   Emerging Competencies

Competency

Key Skill

Storytellers

Explain the business relevance of the data

Artists

Draw pictures and visualize data for greater insight

Behaviorists and social anthropologists

Understand human foibles and idiosyncrasies

Detectives

Passion for uncovering the truth

Philosophers

Able to deal with ambiguity

Jazz musicians and improv actors

Work well with others, read signals and react

Conductor

Able to orchestrate action

Source: Gartner (January 2016)

For a discussion of some of these capabilities, see “Digital Anthropologists Have Important Skills for Emerging Digital Enterprise Strategies.”

Business Software Versus Consumer Software

The consumerization of IT continues to transform business solutions: it forces continual re-examination of technologies, data and design. Consumers have control and access expectations that exceed the limitations of traditional IT. This dynamic blurs the boundary between business and consumer software design, rescopes traditional software distribution, and raises issues of information collection and usage.

The consumer has become an integrated part of the digital ecosystem. Wearables and related apps collect data that drives both behavior (How can I improve/change?) and marketing (What related products and data are valuable to me?). What started in the area of personal health is quickly branching out into personal finance, productivity, well-being and social contribution. Maslow’s hierarchy of needs now has a deep technological dimension. Businesses will take advantage of this digital self by offering new products and services that resonate with individual desires and requirements. But this advantage comes with a caveat: businesses must be allowed, or invited to, participate in the technology ecosystem of their customers, and value must be equally shared without skewing advantage to the company at the expense of the customer. For insight into personal analytics, see “Defining Personal Analytics — Consumers Meet Smart Agents.”

These consumer/business dynamics are parallel to the employee/business relationship. The digital workplace is populated with sophisticated users with consumerlike goals. As companies integrate more with personal information in the context of work, well-designed processes and apps will enhance work products and employee engagement. Processes and apps should make it simple for employees to find information and each other in the context of tasks and relationships. They will also support richer interaction between humans, smart machines, and the IoT. For more details, see “Digital Workplace Graphs Promise to Improve Productivity and Collaboration, but Risks Exist.”

Gartner coverage of consumerization trends includes:

  • “Consumerize Your Enterprise Software Before It Consumes You”
  • “Predicts 2016: Excellent Customer Experiences Hinge on Continuous Digital Experiences”

Who Provides the Technology Platform?

Technology platforms are no longer the exclusive realm of megavendors. The ecosystem of solution providers is diverse and creative, thanks in large part to the availability of cloud-based solutions. Open-source solutions are pervasive and widely adopted, with mature options available throughout the technology stack. In addition, algorithm marketplaces that enable highly flexible technical and business outcomes are forming.

ADAPTIVE SOURCING DRIVES AND HELPS RESPOND TO A CHANGING VENDOR LANDSCAPE

Most of the service providers with which you have traditional relationships have built their businesses on Mode 1 capabilities. They are focused on providing IT organizations with the solutions and services needed to keep their operations running efficiently. Established service providers and vendors are expanding their go-to-market offerings to include Mode 2 innovations, where they will compete head-on with smaller, more nimble providers that are often built from the ground up on digital business principles.

Digital business requires more speed and agility, which means you need a remixed portfolio of providers. Adaptive sourcing is about shorter-term contracts, experimental relationships with vendors, and “techquisitions.”

Gartner research on adaptive sourcing includes:

  • “Adaptive Sourcing Strategies Are Bimodal by Design to Accelerate Digital Transformation”
  • “Techquisitions: An Uncommon Approach Some CEOs Use for Digital Business Acceleration”
OPEN SOURCE IS EVERYWHERE

As shown in Figure 2, open source is widely adopted by businesses worldwide (see “Survey Analysis: Open-Source Software Adoption and Governance, Worldwide, 2014” ). Open source enables lightweight options for app development and deployment (for example, Apache, MySQL and Linux), but also underpins some of the most significant business demands, like advanced analytics (such as Apache Hadoop). The adoption of open source (which has been growing for more than a decade) brings with it a revised approach to governance and developer engagement: some decisions about tool choice lie with the people on the front line, where the benefits of crowd-driven innovation and flexibility can be gained.

Figure 2. Gartner Survey: Open-Source Software Trends and Demand

Research image courtesy of Gartner, Inc.

Source: Gartner (January 2016)

ALGORITHM MARKETPLACES AND CITIZEN INTEGRATION

Algorithm marketplaces are similar to the idea of mobile app stores, which created the “app economy.” The essence of the app economy is to allow completely unknown individuals to distribute and sell software globally without the need to pitch their ideas to investors, or set up their own sales, marketing and distribution channels; they simply benefit from the remarkable infrastructure of those app stores.

The nascent algorithm marketplaces take this one step further. They allow the usage of algorithms and other software components to be brokered. These algorithms are not stand-alone apps, but are meant to be used as distinct building blocks inside specific solutions. The marketplaces will enable discovery and composition of these application fragments by people with various technical and business skills, including citizen integrators working with minimal central IT support. For more on algorithm marketplaces, see “Algorithm Marketplaces Are Bringing the App Economy to Analytics.”

The Changing Nature of Change

  • “The appearance of a new techno-economic paradigm affects behaviors related to innovation and investment in a way that could be compared to a gold rush or the discovery of a vast new territory. It is the opening of a wide design, product and profit space that rapidly fires the imagination of engineers, entrepreneurs and investors, who in their trial and error experiments applying the new wealth-creating potential, generate the successful practices and behaviors that gradually define the new best-practice frontier.” 1
  • — Carlota Perez

Jumping the S-Curve

Most transformations begin with an acceleration at the beginning, followed by a leveling off until the next transformation is discovered. Then the acceleration begins again. During disruptive times (what cognitive psychologists call “breakdown” — where former heuristics stop working), you may jump from one acceleration to the next: the S-curves overlap. This creates a period of ambiguity, and provides an opportunity to search for new patterns.

Breakdown is a remarkably positive mechanism of evolution. It creates a “business as usual” void and opens the door for new ideas to be developed, tested, and implemented. This window of opportunity is, however, quite narrow because the system wants to stabilize as quickly as possible. Now is the time for organizations to turn up the heat on innovation and experiment aggressively. This means bringing some shelved ideas back to life, casting a net for new ones, and creating an environment in which trial and error are equally valuable partners. Consider that one ancient meaning of “error” is “to wander” (from the Latin, “errorem”). When exploring new territory, wandering is sometimes the best way to make discoveries. For more information on heuristics, breakdown, and shifting power, see “Explore a Solution Delivery Perspective for the IT Power Shift.”

Gartner’s bimodal IT and pace layer research is designed to help organizations productively work through these times of disruption, innovation, and experimentation. For more on bimodal IT, see “Bimodal IT: How to Be Digitally Agile Without Making a Mess.”

Divest Applications and Infrastructure

CIOs cannot lead through influence if they are otherwise occupied with keeping everything running. IT departments cannot innovate for the same reason. To enable a focus on digital outcomes, CIOs should take a cue from leading businesses and divest: create a plan to get rid of things that no longer create value or differentiation. This frees the organization up to pursue higher value activities.

Importantly, CIOs should note that divestment is not the same thing as outsourcing. In outsourcing, CIOs let a third party handle a function for them to reduce costs, but the CIO still retains control over that function. In divestment, CIOs transfer the entire process to a third party, on their own platform; that third party can do the job better and the CIO does not maintain control over that function. Joining another ecosystem opens up new benefits that one’s own business applications will not easily provide — for instance, data sharing with other users, or even benchmarking. Divestment can also bring benefits from innovation, agility and speed (see “Rising to the Challenge of Digital Business: Key Insights From the 2015 Gartner Symposium/ITxpo Keynote” ).

A good starting place on the divestment journey is to actually look at what not to divest, including:

  • Your innovation capability
  • Your digital strategy
  • Your differentiating algorithms

Keep those assets of value close, and question everything else. It is time to rebalance the portfolio.

Go for Radical Cloud Adoption

Cloud computing continues to mature and increasingly dominate IT and business conversations worldwide. Hype remains high and adoption is increasing (see “Hype Cycle for Cloud Computing, 2015” ). Across the landscape of cloud computing, however, technologies, services and inconsistently used terms can create confusion (see “The Top 10 Cloud Myths” ). Cloud vendors and users alike make outlandish and misleading claims about cloud capabilities (“cloudwashing”) that can add to this confusion.

There are many examples of organizations that have recognized the benefits and avoided the pitfalls of cloud computing. Understanding the broad landscape of cloud service offerings and technologies, as well as the cloud computing terminology, is critical for organizations looking to exploit the real benefits of cloud computing (stated as “business outcomes”).

Conventional wisdom, longstanding biases and news stories continue to sway views on a daily basis, but the trend toward cloud acceptance is unmistakable. Excitement around its potential to accelerate digital business permeates the market. As this continues, the role of the cloud as a carrier, or vehicle, for digital business is becoming paramount. (For more on the future of the cloud, see “Predicts 2016: Cloud Computing to Drive Digital Business.” )

Cloud computing in 2020 is likely to feature extensive sharing of both applications and data, in the spirit of algorithm marketplaces and the economics of connections (see “Digital Business Gives Rise to the New Economics of Connections” ).

Take It Step-by-Step, Using a Pace-Layered Approach

So, how do you handle all this experimentation along with the constant need to support an operational runtime and extensive portfolio?

Looking at how other industries deal with the problem of variable rates of change in complex systems, Gartner applies the term “pace layers” to the evolution of applications in an organization. Pace layers mirror the concept of “shearing layers” that was developed by Stewart Brand. 2

Brand argues that architectural layers have different “paces” of change, but they must be designed to work together so that the building can function effectively. We believe that this same idea of pace layers can be used to build a business application strategy that delivers a faster response and better ROI, without sacrificing integration, integrity or governance.

In the past, many companies have had a single strategy for selecting, deploying and managing applications. They may have had methodologies for classifying applications by value or technological viability, but they did not recognize that applications are fundamentally different, based on how they’re used by the organization. Gartner has defined three categories — or layers — to distinguish the various business capabilities (and its corresponding applications) that a company needs to manage in order to effectively deliver its business strategy and help IT organizations develop more appropriate application strategies:

  • Systems of Record Usually found in business capabilities with a clear focus on standardization and/or operational efficiency; these are often subject to regulatory/compliance requirements.
  • Systems of Differentiation Typically related to business capabilities that enable unique company processes or industry-specific capabilities; these sustain the company’s competitive advantage.
  • Systems of Innovation — New applications that are built on an ad hoc basis to address emerging business requirements or opportunities; these involve an experimental environment for testing new ideas and identify the company’s next competitive advantage.

These layers correspond to the notion of business leaders having common, different and also new ideas. The same business capability may be classified differently in one company than in another, based on its use and its relationship to that business model. We also expect to see applications move between layers as they mature, or as the business process shifts from experimental, to well-established, to industry-standard.

Begin by experimenting with new ideas and business models in systems of innovation with a bounded audience and budget. Take ideas that work and build systems of differentiation that scale to the whole business, and work your way down. As you gain maturity, innovation moves into the core to support other kinds of activities and the core systems of record will be fully renovated to support the new business models (for more details, see “How to Develop a Pace-Layered Application Strategy” ).

Conclusion

  • “If you have a clear vision of the future ten years hence, you’re a psychotic.”
  • — Richard Rumelt

Projecting the future is a tenuous act at best, insane at worst. In this research, we have attempted to draw some lines of perspective for the near future, based on real patterns and activities that are already appearing now. Change is coming faster than ever, accelerated by continual advances in technology and the acquisition of data. It is a time of experimentation and remixing: we are changing how solutions are delivered and by whom. This requires new levels of trust and influence, and a collaborative approach to innovation at a time when the end results are not always clear. Iterate your way toward that future by closely examining what you have, what you need and where it will come from, while all the time fighting against your own biases and inertia that would hold you back.

Evidence

This research perspective is derived from the work of multiple analysts in Gartner’s Enterprise Software team and beyond. References to the underlying work have been provided in-line and in the Recommended Reading section.

1 C. Perez. “Technological revolutions and techno-economic paradigms.” Cambridge Journal of Economics. 17 July 2009.

2 S. Brand. “How Buildings Learn: What Happens After They’re Built.” Viking Penguin. 1994.

Leave a Reply

Your email address will not be published. Required fields are marked *