AI work assistants require significant guidance

AI tools like Microsoft’s Copilot and Google’s Gemini are supposed to make businesses more efficient, but chief information officers say that these tools often need a lot of data cleaning and management to work well, which shows that they are having a hard time being deployed right now.

Work assisters that use artificial intelligence were created to make it easier for businesses to use cutting-edge technology. That’s not quite what’s happening. Chief information officers assert that extensive internal work is necessary to maximize the effectiveness of the costly tools.

Sharon Mandell, chief information officer of network tech company Juniper Networks, said, “It’s been more work than I thought it would be.” Mandell is trying tools from several vendors but doesn’t feel ready to use any of them yet.

The goal of tools like Copilot for Microsoft 365 and Gemini for Google Workspace is to give corporate workers access to the full power of creative AI. These tools promise safe, pre-packaged ways for businesses to use the technology.

The tools should be able to provide accurate answers to questions like “What are our most recent sales numbers?” when used with the Microsoft or Google business suites and large amounts of enterprise data like emails, documents, and spreadsheets.

But that doesn’t always happen. The enterprise data they are receiving isn’t always accurate or up-to-date, and the tools are still in the process of development.

Mandell told her that if she asks the AI tool a question about data from 2024, it might give her an answer based on data from 2023. The big agricultural company Cargill said that an AI tool got a simple question about who is on the company’s top team wrong.

A tool at Eli Lilly gave wrong answers to questions about spending policies, according to Diogo Rau, chief information and digital officer at the drug company.

The mistakes happen at a time when companies are becoming more interested in generative AI, and CIOs are trying to test the efficiency gains that tools that can cost $30 a month per user can offer.

“I’m still optimistic about AI, and I’m sure we’ll get there.” Mandell said, “It’s just taking a little longer than we thought.”

For CIOs who want to move forward with technology, it is now important to clean up and organize their data so they can get the most out of it.

The vice president and head of the IT group at Hitachi Americas, Bala Krishnapillai, said that the company has seen cases of inconsistent, duplicated, and incorrect data, which creates confusing information for AI outputs.

He said that the company is always adding to and improving its data to make sure that AI tools that use it get the correct results.

In this process, the company’s data experts check and clean up new data, putting it all together in a “golden record” that doesn’t have any duplicate or conflicting information.

Michael Bradshaw, CIO of Kyndryl, said that cleaning up data was the first thing the company did as it got ready to use Copilot. According to Bradshaw, Copilot can do a lot, but a business needs to be ready to use it by keeping data safe, organized, and up-to-date.

Vendors are aware of the issue. “As businesses started to use Copilot, people began to find data that businesses either didn’t know they had access to or realized wasn’t as up-to-date or valuable as it could be.” “They then realized, ‘Oh, we need to do more,'” said Jared Spataro, Microsoft’s corporate vice president of AI at Work.

Challenges of Copilot: Contextual Accuracy and Data Management

The fact that Copilot doesn’t always know where to look for an answer makes things more difficult, according to Spataro.

When asked about revenue, Copilot might not know to go straight to the company’s financial system of record. Instead, it might look at any revenue-related numbers in emails or documents, he said.

Microsoft just released a new tool called Copilot Studio to help companies fight this problem. This tool lets companies tell Copilot to look for answers from trusted data sources within their systems.

Spataro stated that the difficulty of cueing contributes to some of Copilot’s issues. He said that users might not know how much context they need to give Copilot to get the right answer.

He also said that Copilot could improve how it asks for more context when it needs it.

It seems like a lot of people are unhappy with the technology when they first see it, according to Spataro.

Richard Seroter, Chief Evangelist for Google Cloud, said that he thinks companies are doing data management work they might not have done as quickly in the past because they want to use Google Workspace tools like Gemini.

He said, “AI will not be as useful if your data is not in order.” “Buy six units of AI and expect your business to change by magic.”