Gartner identified Applied observability as Top Strategic Technology Trend for 2023. I wrote some ideas “on the back of an envelope” as a co-author of the trend with Frances Karamouzis – who was the original mastermind behind.
As organizations launch new digital products and services, the need for granular visibility into product usage is important not only for operational support, but also for customer success, product management, and support teams. sale. Building observable systems helps business and IT players understand what features customers care about most, what issues they struggle with, and how they can improve service.
Observability also offers a new approach to managing business change within organizations – Instead of monitoring applications and business processes, which are inherently reactive, observability relies on instrumenting processes with data and necessary control mechanisms to enable proactive and preventive actions. Applied observability combines telemetry data obtained across multiple systems to make data-driven business and IT decisions. IT managers must go beyond using observability for system reliability and create an observable digital enterprise.
Customer adoption of new and existing features – this can indicate reasons for customer churn
Demographics of active vs inactive users
Analysis of trends in customer satisfaction and its correlation with service levels and performance
Ability to provide granular insight into enterprise systems and workflows since observability is “built-in”
Observability provides a measure of how well the internal state of a system can be inferred from its external output. Applied Observability extends this principle and applies granular visibility to multiple technology domains such as applications, infrastructure, data, network and security as well as business processes to provide valuable business insights. This ability to leverage observability data helps increase innovation, improve resiliency, and improve customer adoption, engagement, and experience.
Historically, organizations have struggled to plan or learn from past failures due to insufficient data. However, today we have the exact opposite problem – it is not lack but an overabundance of data that makes it difficult to extract the right signals from noisy data. Applied observability involves aggregating, correlating, and analyzing observability data across multiple tiers and technology domains and making that data available to business and technical roles to inform manual and autonomic decisions.
Extend the benefits of observability to multiple roles
Applied observability is about harnessing the value of observability data to meet the needs of different roles in business and IT. For example, application teams responsible for customer experience will benefit from customer observability data. Similarly, I&O teams responsible for managing service levels will need IT observability data such as events, logs, metrics, and traces. Similarly, finance departments managing operational spend may want to know the reason for increased cloud spend and have rich data to make informed decisions.
Observability data is only as valuable as its ability to translate knowledge about the health and performance of individual systems into overall enterprise health and performance. Connecting individual components using telemetry across all layers of business and IT system topology is essential to take full advantage of observability. Key areas of observability underlying this trend include:
Observability of infrastructure, including networks, terminals, compute, storage, communication devices, etc.
Observability of applications including services, APIs, databases, open-source and 3rd party dependencies
Business decision making related to product decisions
Applied observability extends the use of observability to solve business intelligence problems. For example, tracking customer adoption and sustained use of a new feature requires continuous observability of user experience and usage patterns. The ability to get answers to exploratory questions using observability data helps create new insights and fill knowledge gaps for engineering and business stakeholders.
Applied observability makes observability a shared responsibility and a shared practice within the organization.
Valor Points and Failure Points
Observability data by itself has no value unless it augments decision-making at different “value points“. Value points can be thought of as critical junctions in business workflows that affect service levels, user experience, and business decisions. Therefore, it is crucial to ensure that data observability are easy to use in all teams – those that are drive change and others impacted by change.
The application of observability helps to discover discrepancies between how we think systems behave vs how they actually behave.
The increased complexity of distributed system architectures creates the need for end-to-end observability to ensure systems are reliable and resilient. Distributed systems are characterized by several points of failure, complex web of interactions and the inability to predict the impact of component failures on the system (as a whole). As consumers use digital services as the first and in many cases the only point of contact to meet their needs, this greatly increases the need for digital resilience. Optimizing resiliency metrics such as MTTR and RTO requires accurate observability data for troubleshooting and root cause analysis.
Introduce observability as a core ingredient of application design in addition to practices such as observability-driven development. Just like security, integrate observability into the entire software development lifecycle.
Improve the user experience of digital products and services by instrumenting enterprise workflows and applying observability at every layer of the technology stack to discover, learn, and improve the use of IT services.
Enhance skills and equip teams with observability design and architecture capabilities to convert system performance into business performance.