Send us a note or book a short meeting.
77% of SAP companies already use AI, but only 3% of it runs on SAP. The rest runs past their own SAP data with ChatGPT and Copilot. This article shows what Joule can really do today, where it breaks down, and how a third-party agent like the ORAI Agent gets your AI to your SAP data in weeks, not months: through standard APIs, with real user permissions, on-premises if you need it.
Copilot, ChatGPT, in-house LLM setups: in the business units, AI is already part of the day. Just outside SAP. The answer someone needs sits in the ERP, but the AI pieces it together from an office file, with predictably shaky results.
The figures from the DSAG Investment Report 2026 are blunt: 43% of SAP customers in the DACH region have implemented AI use cases. But only 3% of productive AI runs on SAP solutions. 77% rely on non-SAP AI like ChatGPT or Copilot.
That isn't distrust of SAP. It's pragmatism. And it leads straight to the question that defines 2026: how do you get AI to your SAP data, without hallucinations, without compliance risk, and without chaining yourself to a single platform?
Three things shifted in a matter of months:
Joule Studio has been generally available since Q1 2026. For the first time, customers build their own Joule agents, with their own skills, data connections, and business logic.
MCP is now SAP strategy. The Model Context Protocol connects tools to LLMs, inbound and outbound, making agents interoperable across platforms.
Gartner says 40%. By the end of 2026, 40% of all enterprise applications should include task-specific AI agents, up from under 5% in 2025.
The technology is here. The market expects AI as standard. Yet productive use among SAP customers stays low. Why? The implementation.
Joule isn't a chatbot. It's SAP's central AI layer across the entire cloud portfolio. Over 40 standard agents already ship: cash management, bid analysis, production planning, plus Joule for Developers for ABAP and CAP.
Does your use case fit a standard agent, and is your BTP foundation in place? Then you get there fast. For many companies, that's exactly the catch. Five limits come up in project after project:
1. The BTP bar is high. Joule expects a consolidated Cloud Identity Services tenant, a BTP subaccount with Cloud Foundry, SAP Build Work Zone, the Integration Suite, and more depending on the scenario. Build that foundation only halfway, and you never reach production.
2. Standard agents don't know your processes. The cash management agent knows your accounts, but not your escalation rules, your risk assessment, or your special customer agreements. The moment it gets specific, you need your own skills.
3. Bad data, wrong answers. An undocumented custom field, a cryptic Z-table, a master record grown over years: every gap becomes a hallucination.
4. Governance stays your job. Which agent may do what? Which data leaves the BTP tenant? Which LLM for which data class? Joule delivers the technology. You deliver the answers.
5. Your data isn't only in SAP. It's also in Salesforce, the data warehouse, SharePoint, ServiceNow. Standard Joule covers the SAP part. The rest is on you to connect.
These limits leave three paths. The question isn't whether Joule is good, but which path fits your use case.
Option 1: Standard Joule agent. Use case fits one of the 40-plus SAP agents, BTP in place? Fastest path, barely any custom development, fully standard. Strong for classic finance, HR, and procurement processes.
Option 2: Custom Joule agent. You build your own skills and logic on the Joule platform in Joule Studio, including multi-agent and MCP. Strong when the solution is SAP-centric and should live in the Joule UI.
Option 3: Third-party agent outside SAP. You build nothing inside the SAP world. You deploy a finished product that reaches your SAP data through standard APIs: the OData interfaces SAP ships anyway, with every user's real permissions. The agent runs independently of Joule and BTP, connects SAP in one interface with Microsoft 365, SharePoint, CRM, or DMS, and runs on-premises or air-gapped when you need it. That's exactly why we built the ORAI Agent: chatbots and AI agents for SAP and surrounding systems, without ABAP, without touching the SAP system, with your permissions applied 1:1. Strong for fast time-to-value, for knowledge and self-service across multiple systems, and anywhere data sovereignty matters or the BTP foundation isn't there yet.
One question sorts it out: where should the AI live? SAP-centric, inside the SAP UI, BTP in place? Joule. Across several systems, live fast, in your own infrastructure? Third-party agent.
Using the ORAI Agent as the example, in five layers:
SAP access. Standard OData on S/4HANA or ECC. No extra ABAP, no proprietary adapters. Connection via SAP Cloud Connector or VPN. Every user signs in with their own SAP user, so your role concept applies 1:1. Sales sees its region, purchasing its material groups.
Surrounding systems. Microsoft 365, SharePoint, CRM, DMS, through the same interface and their standard APIs. So the agent answers questions that bring SAP data and document knowledge together.
Agent and skills. Skills for recurring tasks, automatic consolidation of data and documents, answers in natural language, with the source and a link to the original document.
LLM. Azure OpenAI, OpenAI, or a local model for air-gapped operation. You choose by performance and data protection.
Operation. Each customer gets their own, isolated instance, running wherever you want it: in the cloud, in your own data center, or fully offline. Every query is tied to the real user, so everything stays fully traceable and auditable, including the evidence for TISAX, ISO 27001, and SOC 2.
The point: fast to production, independent of your BTP maturity, data sovereignty stays with you. The agent uses the interfaces SAP itself recommends, instead of building a second integration layer.
A productive ORAI Agent comes to life in six clearly defined phases. How fast that can go shows in a machinery manufacturer with 800 employees: with the ORAI Agent, sales and service ran on productive product knowledge in around three weeks, with answers in under 10 seconds instead of 20 to 30 minutes.
Phase 1: Use case. Which process pain points should the agent solve? Define the questions, systems, and goal.
Phase 2: Architecture setup. Decide on deployment and LLM hosting: in the cloud, on-premises, or air-gapped.
Phase 3: Agent definition. Define the agent's tools, skills, and workflows.
Phase 4: User training. Involve key users, fine-tune vocabulary and answers.
Phase 5: Testing. End-to-end tests, permissions, and the connected systems.
Phase 6: Rollout. Go-live in production with hypercare and an established feedback loop.
The second use case is then much faster. Connection, permissions, and skill patterns are in place, so the next application builds on top.
AI projects rarely fail on the technology. More often, on these three:
1. Data quality is the prerequisite, not the result. If the master data is wrong, the agent answers wrong. The fix isn't a bigger model, it's clean data.
2. Governance belongs in phase 1, not phase 5. What may the agent do without asking? Which data may the LLM see? Who is liable for a wrong decision? Settle it before you build.
3. Cost is almost always underestimated on the first project. Every LLM call costs. An agent with several tool and LLM calls per request gets expensive fast. Sizing, caching, and the right model mix belong in the architecture.
Plus, DACH-specific: data residency and the EU AI Act. You have two levers: deliberately choose the model region with your provider, or run a local model air-gapped, where the data never leaves your data center. What matters is that you decide actively and document it.
The DSAG figures are clear: AI has arrived in companies. The only question is whether it reaches your SAP data or runs past it. Accept ChatGPT shadow IT as your strategy, and you lose on three fronts: data sovereignty, compliance, and the connection to your ERP core.
For SAP-centric use cases, Joule is the fastest answer, as long as the BTP foundation is in place. If it isn't, if speed matters, or if the data can't leave your data center, a third-party agent closes the gap from the start: the speed of non-SAP AI, but with real access to your SAP data, through standard APIs and real permissions.
The most expensive mistake would be to wait. The technology is mature, the competition is already building. Climb the learning curve of data connection, governance, and use-case design now, not when management suddenly wants a productive case.
We combine SAP depth with real AI experience, from Joule and the Generative AI Hub to connecting SAP to your surrounding systems via API. We find the right use case and architecture with you, build the data connection cleanly, and bring the first productive agent home in a few weeks, with governance, cost model, and enablement.
When a finished product is the fastest route, we bring the ORAI Agent: SAP connected to your surrounding systems via standard OData, permissions 1:1, in the cloud, on-premises, or air-gapped.
In 30 minutes, we'll find your fastest AI lever and the architecture for it. Free, concrete, no sales pressure.
