Connect with us

Hi, what are you looking for?

Business

Q&A: Impact of AI and ML on data management (Includes interview)

Data management is vital for modern businesses, covering the process of collecting, storing, organizing and maintaining the data created and collected by an organization. Due to the complexity, some organizations are turning to new technology as part of their digital transformation initiatives. The increase use of artificial intelligence and machine learning with data management processes will require a reassessment of infrastructure and for companies to select the write types of systems and to get to grips with process reengineering.

To understand more about the application of artificial intelligence to advance data management practices, Digital Journal caught up with Walt Kristick, SVP of applied and advanced technology at APEX Analytix.

Digital Journal: What are some groundbreaking applications for AI/ML as it relates to more effective data management?

Walt Kristick: Today’s companies have unprecedented access to massive amounts of data — but one question remains, how can they use this data to add real value for their clients? With AI and machine learning, we’re beginning to see companies shift their focus from data quantity to data quality as they try to operationalize and deliver a greater ROI on their data management solutions.

To better manage vast amounts of data and add measurable value for clients, companies are now using AI and machine learning to extract insights that help clients better manage risks to their supply chain. For example, IBM Watson Discovery News is an AI tool that can be used to analyze 300,000+ articles per day from around the world — looking for trending topics or any negative news on a client’s suppliers. This technology then scores these articles, including a sentiment score and a score associated with the likelihood an article falls into pre-identified categories of risk. This analysis enables organizations to identify risks faster than ever before, and to put their clients in a position to proactively mitigate supplier risk versus reacting to a crisis.

With insights mined by AI and machine learning, companies can also answer questions like, how do I consolidate my suppliers? How can I optimize payment terms across my suppliers? Knowing these answers allows them to operate their supply chain with greater efficiency.

DJ: Will AI/ML solutions finally deliver the promise of autonomous, self-managing databases?

Kristick: Managing storage and computing resources, tuning for performance enhancement, indexing, data organization, compliance and security are crucial components in present-day data management. When the data managed is streaming or heavily dynamic, lots of critical and time-sensitive decisions must be made when receiving, diverting into storage, processing, sharing, serving for applications, etc. Traditionally, these decisions are carried out by the rules managed by DBAs and other data/application specialists. In the past decade, AI/ML technologies showed great success in decision making applications solving the pain points in classification, natural language understanding, complex pattern recognition, anomaly detection and predictive analytics. These same techniques can be transformed into data management solutions to achieve autonomous, self-managing databases.

DJ: What will be the impact of AI/ML on data management jobs like database administrators and data analysts?

Kristick: It’s easy to understand why advances in AI and machine learning may give the impression that data management jobs will be replaced. However, these technologies are only one part of the data management process. The roles of data analysts and database administrators will shift and expand to include new responsibilities, but they certainly won’t disappear.
Where data management jobs can add real value is by going beyond finding insights (something which AI and machine learning can do), to spending more time interpreting this information and answering the question ‘what’s next?’ Identifying trends and contextualizing data will be increasingly important tasks for people in data management roles.

Graphic showing the different ways that numerical data is expressed in different cultures (Barbican ...

Graphic showing the different ways that numerical data is expressed in different cultures (Barbican Centre, London).

DJ: What are some risks or challenges in running data environments on AI/ML?

Kristick: In traditional architecture, an AI/ML system consumes data environments to fulfill the data needs. Here we discuss a slight change to that architecture by letting AI manage the data environment. This transition must be done smoothly in order to prevent catastrophic failures in the existing systems as well as to implement robust systems. AI/ML systems being developed to manage data environments must be considered as mission critical systems, and the development must be carried out very carefully.

Since data is the driving force of present-day business decisions, data environments will be the heart of the business. Therefore, even a slight failure in data management will incur a significant cost to the business by loss of operational time, other resources and user trust. One other challenge is to fill the knowledge gaps of data professionals and AI/ML experts in the areas of AI/ML and data management, respectively.

DJ: How is the rise of AI/ML in enterprises shifting expectations of data management departments/staff?

Kristick: We see first-hand the benefits of AI and machine learning on streamlining organizational processes, and in turn, employees get to focus on more valuable aspects of their role. For data analysts, this means more time spent interpreting data and data storytelling. Take for instance the IBM Watson tool. It would be impossible for our staff to analyze thousands of news articles each day to identify risks to our clients’ AP departments. Instead, data management staff can take the insights gleaned from AI and machine learning, and work to identify next steps for addressing these risks. Going back to how AI and machine learning will impact data management roles, the interpretation of this data and creating models to aid decision making will be key expectations of these departments moving forward.

Data is the new oil.

Data is the new oil.
Chiffre01 (CC BY-SA 4.0)

DJ: Are current data environments ready to deliver and support AI/ML initiatives? What are the challenges?

Kristick: Uninterrupted support from data environments is crucial for AI/ML systems in different stages, such as training, deploying and serving. The requirements vary based on the stage and the application. Complexity of data management in traditional data environments and performance bottlenecks in processing large data volumes to train AI/ML systems create some challenges to the on-premise, enterprise level AI/ML systems. Newly emerged cloud computing environments like Snowflake, Amazon Redshift and Azure Data Warehouse provide a promising infrastructure to deliver and support AI/ML systems. However, they also possess some challenges. Examples of challenges include infrastructure bottlenecks in the data transfer and establishing secure, reliable, compliance ready services.

DJ: What skills should data managers/admins and analysts emphasize?

Kristick: There’s mostly consensus on the skills necessary for success in data management roles. For example, getting up to speed on statistical programming languages like SQL, R or Python is essential, especially for analysts working with Big Data.

While data cleaning and preparation will continue to be a necessary skill for database managers, it’s important to remember skills like data visualization, critical thinking and strong communication will become even more critical as AI and machine learning take over more of those repetitive tasks. For this reason, I would advise honing in on skills related to being a strong data interpreter and data storyteller in order to continually add value in your role as a data analyst or data manager.

Avatar photo
Written By

Dr. Tim Sandle is Digital Journal's Editor-at-Large for science news. Tim specializes in science, technology, environmental, business, and health journalism. He is additionally a practising microbiologist; and an author. He is also interested in history, politics and current affairs.

You may also like:

Tech & Science

Under new legislation that passed the House of Representatives last week, TikTok could be banned in the United States.

Social Media

Wanna buy some ignorance? You’re in luck.

Life

Platforms like Instagram and Pinterest often suggest travel destinations based on your likes and viewing habits.

Social Media

From vampires and wendigos to killer asteroids, TikTok users are pumping out outlandish end-of-the-world conspiracy theories.