“AI Is Fast Becoming the Raw Material for Other Innovations” Says Balvinder Kaur Khurana

By S G Rickman On Nov 8, 2022

Balvinder Kaur, Data Architect at Thoughtworks has many feathers to her hat. Having started her career as a Software Engineer after graduating in 2007, she has come a long way in leading Data Engineering teams and has been instrumental in deploying data strategies for Thoughtwok’s clients helping them chart a road map for milestone-based plans. She is engaged in delivering Tech-talks, participating in fresher and lateral interviews, and takes a keen interest in designing training courses and mentoring freshers. She is an active member of I-Care, the CSR wing of ThoughtWorks, and Spice, its event planning committee. Analytics Insight has engaged in an exclusive interview with Balvinder Kaur Khurana, Principal Consultant, and Data Architect, Thoughtworks.

1.Tell us how Thoughtworks is contributing to the AI and ML industry of the nation and how the company is benefiting the client.

Thoughtworks is known for custom software delivery, and we have been the trusted advisor/partner for clients for decades now. And, this kind of partnership continues with the growth of the AI and ML industry. In the space of AI and ML platforms, Thoughtworks is defining and applying the best practices, and working with bleeding-edge technologies in AI/ML deliveries. Thoughtworks has worked on some leading AI/ML initiatives in the industry. Here are a few:

1.In January 2022, Open Network for Digital Commerce (ONDC), a Government of India entity, conducted a global hackathon to enable and promote local digital commerce across different industries in India. A total of 63 teams from around the world competed to build solutions for the retail industry to facilitate the participation of small retailers, vendors, and kirana stores in online commerce including five teams from Thoughtworks. In the results that were announced last week, ONDC recognized the team from Thoughtworks as winners for their solution to break India’s diverse language barrier and allow users (buyers, sellers, logistics service providers) to participate in trade.

2.OpenNyAI – OpenNyAI is a mission aimed at developing open-source software and datasets to catalyze the creation of AI-powered solutions to improve access to justice in India.

3.Thoughtworks has helped to create an NLP benchmark dataset (called BUILDNyAI) which provides structure to Indian Court Judgements using sentence rhetorical roles. This benchmark would help to collaboratively create open solutions for the automatic structuring of court judgments which will act as a foundational block to enable the creation of Legal AI solutions. This work is part of the OpenNyAI mission.

4.Thoughtworks created a scalable speech recognition platform to unleash artificial intelligence (AI) technology in India. This platform has the largest collection of languages and uses speech technology along with deep learning.

Apart from the above-mentioned projects, Thoughtworks has been proactively contributing to AI and ML industry globally. We recently had Fourkind join the Thoughtworks family. Fourkind was founded in 2017 with the mission to make a real change in our world, and for the clients, they work with. They specialize in data science, machine learning, strategy, design, and engineering. Fourkind is known for its many successful projects, including optimizing the big airport in Finland and creating the world’s first AI-generated whiskey for a Swedish Client.

2.How are AI and ML evolving today in the industry as a whole? What are the most important trends that you see emerging across the globe?

An Analytics team that generates reports on demand is becoming obsolete. Rather, it is the team that continuously generates insights and quickly automates the desired analytics that is becoming the much-needed next evolution in the space.

The industry is also ready to pay for data that is useful – to generate insights from and to the benefit of customer bases and markets, especially if these insights provide an edge over competitors. This strategy is fuelling a lot more AI, and ML-based work within organizations and growth for third parties that aggregate data.

We expect further AI/ML adoption to deepen value for businesses and the ethics analysis of probable solutions to be a part of AI/ML exploration and implementation. We see a focus on ML Ops and model monitoring.

Explainability, Interpretability, Causality – a lot of business problems require causal interpretations so that interventions can be carried out.

Efficiency: We are seeing a rise in the approach that involves less compute to ensure the data gathered is relevant and efficient, and not overwhelmingly voluminous. There is a growing exploration of methods that can learn from less data and/or less compute.

Role specialization: we expect new, specialized roles emerging out of the catch-all data scientist descriptio

3.What are the key trends driving the growth of AI and Machine Learning in digital customer experience?

AI is now available at users’ fingertips – from recommendations on shopping to alerts for proactive health checkups, from targeted news to enhanced virtual backgrounds. This has significantly helped enhance the digital experience for customers.

Interestingly, AI is becoming the raw material for other innovations. A lot of these innovations have become so seamlessly embedded that it’s no longer called AI.

Real-time decisioning, predictive engagement, and edge computing have made sub-millisecond decisioning possible. These are decisions based on the most recent data, such as data from a current interaction that a customer is having with a business. AI-driven chatbots are increasingly becoming a part of the digital customer experiences for most businesses – to the extent that customers are also now considering such chatbots trustworthy.

Hyper-personalization and Conversational AI provide personalized experiences that are also much quicker and more convenient over traditional ways of interacting with businesses. They give customers a more personalized experience and eliminate the pain points in a customer journey.

4.What role has Thoughtworks played in the innovations of new technologies?

Thoughtworks brings rich expertise to help uncover patterns with powerful analytics and machine learning. A lot of our clients need help in prioritizing and finding the right problem to solve through data. It is critical to pick a problem that is feasible and also the most impactful in achieving the business goals.

We conduct Data and Problem Discovery with our clients to help them find the right problem to solve. This involves close collaboration with the client and understanding of their short and long term goals. We understand the KPIs the client organization is tracking against these goals and analyze the data to prove the problem actually exists.

We conduct workshops with the client to discover the possible root causes of the problem and again using data and knowledge of the business we drill down to the cause which, when handled will solve the problem. This process also involves deciding on a business metric, which becomes a focal point for finding a solution.

We ensure that our teams explore both AI and non-AI ways to solve problems. In case we discover that a process or system change (non-AI) solution is not sufficient or timely, we build a solution hypothesis and use the data to produce a baseline AI solution. This solution has to have a metric that can be easily measured and is connected to the business metric. It is critical at this point that we also learn about the cost of error or inaccuracy in the AI solution we are proposing. Testing this solution with a sample of users and obtaining feedback helps us ensure that we learn early and evolve quickly or, we fail fast in case our solutions do not work. This entire process from Data and Problem Discovery to Solution Hypothesis enables the clients to be data-driven.

5.What role has Thoughtworks played in the innovations of new technologies?

Thoughtworks is a thought leader when it comes to architecture principles and delivery practices, for software, data platforms, and AI and ML projects.

Some of the areas where we are leading with thought leadership and equipping the tech community with the tools to further the AI and ML space are:

1.CD4ML

CD4ML helps to maintain productivity, collaborate effectively, and continuously and seamlessly deliver value in practice.

Delivering Machine Learning and AI systems bring extra complexity, as they are harder to test, explain, reproduce, and improve. It also requires coordination between different disciplines, ranging from Data Engineering, Data Science, Testing, Infrastructure Engineering, and Release Engineering, while aligning this with business needs.

Continuous Delivery for Machine Learning (CD4ML) is a software engineering approach in which a cross-functional team produces Machine Learning applications based on code, data, and models in small and safe increments that can be reproduced, retrained, and reliably released at any time, in short, adaptation cycles. This is a foundational capability for organizations to become data-driven, especially in highly regulated industries. The absence of this will present a scaling challenge and prevent them from getting the most out of their investments in ML.

The industry is starting to coalesce around terms such as MLOps and DataOps (similar to DevOps), which highlights the needs for operationalizing Machine Learning and Data systems. CD4ML is ThoughtWorks’ approach to tackling these problems, leveraging our pioneering experience with Continuous Delivery.

2.Data Mesh

Data Mesh is an analytical data architecture and operating model where data is treated as a product and owned by teams that most intimately know and consume the data. Today, data is ubiquitous. Data is the by-product of any and every digital action we take. Everything, every system, every process, and every sensor generates data. Technology makes it easier for organizations to collect and store data, and for businesses to leverage it to make better decisions or create more tailored experiences for their customers.

However, organizations are struggling to enable and empower their employees to make the most informed and timely decisions possible. Centralized data platform architectures fail to deliver insights with the speed and flexibility scaling organizations need. Data Mesh serves as a solution to this problem.

Data Mesh applies the principles of modern software engineering and the learnings from building robust, internet-scale solutions to unlock the true potential of enterprise data.

We follow the four principles of Data Mesh

Domain ownership – reducing the hops between analytical data consumers and data sources.

Data as a product – applying design thinking for data assets. Encapsulating related code, policies, and infrastructure in a cohesive product.

Self-serve data platforms – removing friction and technological complexities from the interaction between data producers and consumers.

6.What are the trends expected in 2022?

We expect the following trends to take place in the industry:

1.There will be more emphasis on model interpretability and monitoring. With less reliance on end-to-end models

2.Cloud-agnostic ML platforms will become more common

3.MLOPS is a significant area of focus with several startups innovating in this space

4.Due to the environmental impact and the amount of compute required for Deep Learning models, there is a growing emphasis on efficiency. Smaller models with performance comparable to huge ones will be sought after

5.Graph Neural Networks are an area of focus in Deep Learning due to current advances alongside their suitability for a variety of applications

6.MultiModal learning is also a growing topic where NLP, Speech, Vision, etc. are all used as Inputs to the same model. This helps the model use information and context from different modalities and ensures great results.

The post “AI Is Fast Becoming the Raw Material for Other Innovations” Says Balvinder Kaur Khurana appeared first on Analytics Insight.