AI Assessment in Organizations

Evaluate, Benchmark and Strategize your AI Readiness

Artificial Intelligence (AI) is transforming how we live, work, and do business. From smartphones to smart factories, AI is ubiquitous – and whether we realize it or not – it is influencing our everyday interactions with businesses. AI has disrupted our lives by entering the entire gamut of areas like recommendations during ecommerce purchases, personalized video streaming options, predictive texts while typing, and prioritizing social media feeds. Artificial intelligence (AI) has been touted as a means for organizations to cut costs and enhance their quality of services, coordination, productivity, and practice efficiencies. Today, AI technologies are being increasingly used in diverse organizational practices, creating new types of human-machine configurations and playing an increasing role in contemporary organizing. Examples of organizational use of AI can be found in such diverse areas as management decision-making, manufacture, and design. The AI systems involved can be described as rational agents that autonomously respond to inputs—with little or no user intervention—by performing tasks guided by their underlying models and functions. In this manner, AI technologies constitute a new type of agency in the context of contemporary organizing.

As organizations move on their AI journey, they have to infuse AI in their enterprise applications and sometimes build new AI offerings from scratch. The use cases for AI infusion span industries and product categories, and cover a wide spectrum ranging from providing insights, recommending the next best actions, predicting, and forecasting business outcomes to fully automating business processes. Most organizations start their AI journey and soon want to scale AI but get stuck during the process. Identification of all AI capabilities, technologies, client adoption levels, and value derived from AI are some salient metrics that govern the journey’s success.

The history of AI is as interesting as its use cases. In his 1950’s work Computing Machinery and Intelligence, Alan Turing (1912–1954), who is considered by many the father of Artificial Intelligence, laid out the following question: Can machines think?

This question, despite its short length and old origin, remains a frequent source of discussion, navigating the frontier between technology, philosophy, neuroscience, and theology.

Despite AI technologies’ proliferation, accessibility, scalability, and ease-of-use organizations are still struggling to reap their full potential. Industry reports suggest that while the business world is beginning to harness these technologies and their benefits, fundamental transformation barriers remain. For organizations to reap the full potential of digital technologies in general and AI in particular, they need to enable mutual adaptation of technology and organization. However, digital technologies such as AI are said to be notoriously challenging and dynamic, as their adoption entails multiple, continuous, and simultaneous adjustments of organizations’ resources, staffing, culture, and decision-making. A challenge for organizations adopting AI in their operations is that AI platforms vary in both scope and complexity, which hinders familiarity with them and hence their deployment to obtain competitive advantage. This springs partly from the ‘black box’ nature of the algorithms (sets of digital instructions implemented to achieve defined goals) dictating AI responses, which are difficult to understand for members of organizations that are being increasingly shaped by AI. AI platforms are likely to transform organizations in qualitatively different ways from other technologies, so it is crucial to develop an understanding of organizations’ abilities to meet these challenges (their AI readiness).

Since AI technologies have human-like cognitive capabilities, including knowing, learning, perceiving, sensing, acting, communicating and reasoning, their deployment may have far-reaching consequences for organizations and various associated ecosystem actors, including consumers, vendors, frontline service providers, and other stakeholders. However, there is a huge gap between the AI hype touted by AI vendors and its actual use in organizations. Thus, the importance of organizations’ AI readiness in ongoing digital transformation has been recognized.

AI Assessment Framework

“Good questions inform, and great questions transform”, The Proximity Principle (Ken Coleman), is the foundation of any assessment framework.

What organizations want to know,

How can we use Artificial Intelligence to improve our value offerings?
How will AI deployment improve our business?
How do we start with Artificial Intelligence?

AI readiness is an organization’s abilities to deploy and use AI in ways that adds value to the organization. A number of research papers have been published on AI assessment/ readiness frameworks. These frameworks assess and visualize multiple dimensions of the readiness: technologies, activities, boundaries, and goals. These frameworks grade the organization’s capabilities (current and future potential) in each dimension. It highlights the risks and challenges involved in firm-wide mobilization of AI technologies for point solutions to enterprise-wide deployments. The below adoption curve from Deloitte best summarizes the relationship between the scope and the potential value of these AI projects.

Why should one take these AI Assessments?

Understand where the AI industry as a whole is on their journey.
Learn where your organization stands in comparison to the AI industry benchmark.
Uncover what steps you can take to advance your Al initiatives.

With the experience of scoping and implementing multiple AI projects, there are a few key things that organizations can do to ensure their success at deployment, regardless of where you are in your AI journey. This includes identifying the right problem to solve (the Goldilocks Principle of choosing the right porridge bowl), building the proper organizational structure, and using high-quality training data. The entire gamut of responsible AI is also very critical for organizations to consider as part of their project identification and implementation journey.

By taking the AI Readiness assessment, companies can discover which of the four stages their company falls into – explorer, enthusiast, expert, and evangelist. From here, companies can see what percentage of other companies are in that stage, what characteristics put them into that stage, and, most importantly, discover what to focus on to advance your company’s level of AI readiness. But more on this in the coming blogs from my colleagues in NASSCOM DS&AI team.

Industry Benchmarking

Industry benchmarking is an important component of any assessment activity. Getting an insight on how your peers are doing in the same industry or comparable industries is the best way to identify the journey that any organization has traversed till now and what steps are left. A lot of assessments tools exist in the market today which will try to assess any organization’s current state and give some visibility on how the industry peers are doing. Most of these “industry benchmarking” data is provided by OEMs of various AI tools and hence need to be taken in the “right spirit”. The advantage of an organization like NASSCOM providing the benchmarking data is that the data would be less biased and would come from a larger base of respondents.

In the last two years, most AI Readiness benchmarking study puts a majority of companies in the scaling stage. This indicates that most companies are working on increasing the initial identified value of AI and adding more resources and focus to AI programs.

Companies in the scaling stage often see the following:

AI is becoming critical to the business
Investment in AI is consistent and increasing
Executives may have participation and visibility of AI initiatives
Corporate sophistication is increasing, with more teams working on AI programs and starting to think about bias, ethics, or risk management.

How do various tools calculate the AI Readiness?

The next couple of blogs from NASSCOM talk about what factors are considered in building an AI Maturity Assessment Framework. These factors include but are not limited to,