In some ways, the experience at a high-end restaurant is not unlike that of an enterprise that’s embracing generative AI. When a diner is served their food, they see a delicious meal presented nicely and ready to eat, giving little if any thought to the chaos on the other side of the kitchen door that resulted in their dinner.
Similarly, corporate executives and employees using GenAI tools that make their work and lives easier and more productive likely don’t understand the complexities and challenges the AI developers have to navigate to create those time-saving and efficiency-gaining applications.
A report issued today by IBM and business intelligence company Morning Consult puts a spotlight on the stresses generative AI software developers face trying to adapt to and get control of a rapidly emerging technology that is evolving at a pace that is difficult to keep up with.
The study, based on a survey of more than 1,000 enterprise AI developers in the United States, tells a story of programmers struggling with skills gaps and tool sprawl while trying to manage a steep learning curve in a highly complex environment that comes with fast innovation cycles, few standardized processes, and corporate leaders who underestimate their challenges.
“If you’re looking at generative AI over the past two years, the market started with exploring and investigating GenAI, looking for ‘aha’ moments,” Maryam Ashoori, senior director of product management for watsonx.ai, an AI studio that is part of IBM’s watsonx AI and data platform, told The New Stack. “Over the last year, we saw the majority of the market — I’m especially talking about the enterprise — moving towards production. When you think about production and scale … there exists a very complex, complicated AI stack that developers need to harness to be able to deliver on the potential of GenAI.”
GenAI a Development Challenge
Developers’ struggles with generative AI aren’t new. Last year, Atlassian and DX, in their State of DevEx Survey, reported that executives see AI as a key tool for improving developer productivity and satisfaction, even though two-thirds of developers surveyed said they hadn’t seen any real productivity gains by using the technology.
With its survey, IBM took a slightly different tack, looking to find how those programmers creating AI applications and tools are faring in a world of high demand and accelerated innovation. There were 10 challenges listed in the results, and the top six were separated by only a few percentage points, with 33% listing the lack of a standardized AI development process and developing an ethical and trusted lifecycle as the top two.
In addition, 32% mentioned the need to enable customization while 31% pointed to the rate of change within GenAI.
Navigating the AI Stack
The AI stack is highly complex and getting more so all the time, Ashoori said. It involves the infrastructure that includes GPUs from multiple suppliers and is often found in different environments — one day, the developers may use infrastructure in Amazon Web Services (AWS) and the next the IBM Cloud or Microsoft Azure.
There also are myriad large language models (LLMs) — the foundation of GenAI — that developers need to switch between, and now there are AI agents, the latest step in the technology’s evolution that involve pieces of code that can work autonomously and collaboratively to take the necessary steps to solve problems.
However, there are multiple frameworks, tools, and models an AI agent will have to connect to via various APIs, and developers will have to integrate, maintain, and master them all, Ashoori said. In addition, the AI application will have to integrate with legacy systems and existing workflows if they’re going to deliver value. There are a lot of integrations and choices a developer must orchestrate.
Standardization Is a Big Need
This type of environment screams for standardization at every layer, she said.
“Startups are putting something out. The major companies are putting something out,” Ashoori said. “It’s complicated and overwhelming and frustrating for developers, especially the category that call themselves app developers, not the data scientists.”
That’s an important distinction as the paradigm shifts from app development to AI development, she said. Historically, traditional machine learning involved ML engineers and data scientists who really understood it all. However, with GenAI, the burden is shifting to app developers, who have to understand the complexity that comes with the technology, from transformers to neural networks to AI frameworks. However, less than a third — 24% — of app developers said they were experts in GenAI.
“They are using this thing, but basically it’s behind an API,” Ashoori said. “They don’t really understand what’s going on behind the scenes.”
However, because of the evolving nature of GenAI, they need that understanding, which will give them the flexibility to customize their applications. In addition, AI agents are the next big thing in the field, and programmers need to move beyond APIs and use AI frameworks, and there are a lot of them.
For every framework, “if something changes, the developers are expected to update everything in response,” she said. “Because of this, the number one challenge most agreed upon was the lack of a standardized process when it comes to building AI applications. When it comes to AI applications — massively changing every day — it’s extremely complicated and complex.”
Fewer and Easier-To-Use Tools
Similarly, developers said they need tools that are easy to use – and fewer of them. Of those surveyed, 72% said they are using five to 15 tools to create an AI application, with 13% saying they are using more than 15.
They also want the quality of the tools to improve. The top four essential traits enterprise AI development tools need are performance, flexibility, ease of use, and integration with existing tools, the developers said. Those are also the top qualities that are most missing from tools, according to the survey.
Ease of use is a key issue for developers. Only 33% are willing to spend more than two hours mastering a new AI tool, with 42% saying they will spend between one and two hours. Another 4% said they’ll take no more than 29 minutes. In addition, there is a hesitancy to experimenting with new AI tools: 77% said they’ll do it every one to six months. Only 21% said they investigate new tools every month, while 2% said they rarely or never do.
That said, they are embracing a range of various tools to help streamline the development of AI apps. More than half of developers are using low-code (65%) or no-code (59%) development tools to reduce the amount of coding they need to do. In addition, 73% are using traditional pro-code tools that require them to write code in programming languages.
Looking to AI To Help
They also are leaning into AI-based coding assistants, with 78% saying they’re using them for AI software development often or very often, and another 15% doing so occasionally. And those coding assistants are working, with 41% saying they’re saving one to two hours a day and 18% saying they’re taking back three to four hours.
As noted, AI agents are the new kids on the block, and not only do they add more complexity to the development mix, but they’re also helping to fuel the challenges around transparency and the ethical use of AI, Ashoori said. For much of 2023 and into 2024, enterprises were experimenting with GenAI with little concern about regulations or production, though in highly regulated sectors, ethical and trusted AI has always been a big deal.
“Moving forward, even in a non-regulated environment, ethical transparency and traceability of actions is even more important when we think about agents,” she said. “Historically, LLMs were just creating some content input [and] output. That’s it. Now, with agents, LLMs are connected to a series of tools that can take actions on behalf of you. Because of that, it’s essential to have transparency and traceability of these actions. This is not easy.”
There are few AI solutions now that are mature enough for such agents as observability agents, much less the more complicated ones. That’s just the nature of a field that is evolving rapidly, and for AI agents, transparency is key.
“Agents and LLMs should not be just a black box that you get, and you just use it to connect it to the rest of your workflows,” Ashoori said. “You really need to understand the consequences of what you are introducing to your solutions.”
Simplification Is the Cure
The key word for addressing the challenges of AI developers is “simplification.” That includes simplifying the AI development stack and the AI development lifecycle, as well as making tools easier to use and reducing the number needed. It’s a situation that IBM is trying to address through such efforts as supporting open source AI stacks to ensure better transparency, trustworthiness, and innovation as well as watsonx.ai and its collection of tools, frameworks, and integrations, watsonx Code Assistant to augment developers’ skills and automate development processes, and open Granite models.
Still, it will take a village to ease the struggles of AI developers, Ashoori said.
“The whole market collectively needs to tackle this,” she said, noting the AI Alliance IBM and Meta launched in late 2023 and that includes other vendors as well as researchers, developers, and government agencies. “This openness in terms of being open about the technology is allowing us to have a conversation with the market and making it accessible to open source is allowing us to put it in the hands of a wider range of developers to get feedback on the holes and the gaps that the technology has and some of the difficulties. [It’s] also getting the help of the same community to collectively build those. So, it’s not that one that one company is going to do that. But the goal is to surface some of the challenges and the actions that we are taking to encourage and collectively help everyone to build up on those and fill those gaps.”
YOUTUBE.COM/THENEWSTACK
Tech moves fast, don’t miss an episode. Subscribe to our YouTube
channel to stream all our podcasts, interviews, demos, and more.
