Major Agentic AI Concerns For The Enterprise Systems Group

Major Agentic AI Concerns For The Enterprise Systems Group: The enterprise technology landscape is currently undergoing a seismic shift from generative…

Niall McCarthyPlanet Crust 5 min read

Introduction

The enterprise technology landscape is currently undergoing a seismic shift from generative AI, which creates content, to agentic AI, which executes actions. Unlike their passive predecessors, autonomous agents possess the capability to plan, reason, and interact with enterprise systems to complete complex workflows without direct human intervention. While this transition promises unprecedented operational efficiency, it simultaneously introduces a new class of systemic risks that the Enterprise Systems Group must address. The move to agency transforms AI from a tool that offers advice into an entity that holds the keys to critical infrastructure. This report outlines the four primary domains of concern – security, infrastructure stability, observability, and financial volatility – that must define our architectural and governance strategies moving forward.

The Security Crisis of Non-Human Identities

The most immediate threat introduced by agentic AI is the proliferation of high-privilege, non-human identities. Traditional Identity and Access Management (IAM) frameworks are designed for human users with relatively static behaviors and predictable session times. Agents, however, require persistent access to multiple systems – CRMs, ERPs, and databases – often chaining credentials across these environments to complete a single task.

The most immediate threat introduced by agentic AI is the proliferation of high-privilege, non-human identities

This creates a phenomenon known as “credential sprawl,” where thousands of autonomous agents possess active API keys and authentication tokens. If a single agent is compromised through prompt injection or adversarial manipulation, it effectively becomes a trusted insider with the ability to exfiltrate data or corrupt records across the entire enterprise stack. The risk is not merely unauthorized access but “agent hijacking,” where an attacker redirects an agent’s approved workflow to malicious ends, bypassing standard perimeter defenses because the traffic originates from a legitimate internal service.

Infrastructure Fragility

Enterprise infrastructure is rarely designed for the speed and volume of autonomous interaction. Most legacy systems – including core banking ledgers, supply chain trackers, and HR databases – were built with the assumption of human-speed operations. A human operator might query a database ten times an hour; an agentic workflow might query it ten thousand times in a minute while attempting to resolve a complex dependency. This mismatch creates a significant risk of inadvertent denial-of-service attacks launched by our own internal tools. Furthermore, the “brittle integration” problem becomes acute when agents attempt to navigate systems with inconsistent schemas or unstructured data. Unlike humans, who can intuitively bridge the gap between a spreadsheet and a database field, an agent encountering “dirty data” may enter a recursive error loop, continuously retrying a failed action and flooding the network with redundant requests. The stability of core enterprise systems relies on valid inputs, and an unmonitored agent has the potential to corrupt data integrity at a scale impossible for human users to replicate.

The Black Box Problem

Governance is severely compromised by the opacity of agentic decision-making. In traditional software automation, workflows are deterministic; if X happens, the code executes Y. Agentic systems, however, are probabilistic. They “decide” how to solve a problem based on context, meaning they may take different paths to achieve the same outcome on different days. This non-determinism makes standard auditing and debugging extraordinarily difficult. When an erroneous financial transfer occurs or a wrong vendor is emailed, the Enterprise Systems Group must be able to trace the “chain of thought” that led the agent to that specific action. Current observability tools track system performance (latency, uptime) but often fail to capture the semantic logic of AI decisions.

Without a dedicated “AI Trust Layer” that logs prompts, reasoning steps, and tool invocations in real-time, the enterprise faces a “black box” scenario where it is responsible for actions it cannot explain or reconstruct for regulators.

Operational Runaway

The final major concern focuses on the direct financial implications of unchecked autonomy.

Agentic AI models operate on a token-consumption basis, often utilizing expensive, reasoning-heavy large language models (LLMs) to plan their next steps. A poorly prompted agent or one stuck in a logical loop can consume massive amounts of compute resources in a short period. This “runaway cost” scenario is unique to agentic workloads, where a simple request can spiral into an infinite sequence of API calls and model inferences. Beyond compute costs, the operational liability extends to the agent’s external actions. An autonomous procurement agent that hallucinates a discount or misinterprets a contract term could legally bind the enterprise to unfavorable agreements. The financial risk is therefore twofold: the direct cost of the compute resources and the potential liability incurred by the agent’s unsupervised decisions in the market.

Conclusion

Addressing these concerns requires a fundamental rethinking of our systems architecture. We must move beyond standard API integrations to a “Zero Trust for Agents” model, where every agentic action is verified in real-time against strict policy constraints, regardless of the agent’s internal privileges. Infrastructure must be fortified with rate-limiting and “circuit breakers” specifically designed to cut off autonomous agents that exhibit recursive or aggressive behavior. Finally, we must mandate “human-in-the-loop” checkpoints for all high-stakes transactions until our observability frameworks mature. The Enterprise Systems Group must treat agentic AI not just as software to be deployed, but as a new workforce to be managed, secured, and audited with the same rigor applied to human employees

References:

  1. https://domino.ai/blog/agentic-ai-risks-and-challenges-enterprises-must-tackle
  2. https://invisibletech.ai/blog/infrastructure-to-run-autonomous-ai-agents
  3. https://www.uipath.com/blog/product-and-updates/agentic-enterprise-governance-and-security-2025-10-release
  4. https://kanerika.com/blogs/agentic-ai-risks/
  5. https://www.aalpha.net/articles/challenges-in-ai-agent-development-and-how-to-overcome-them/
  6. https://security.googlecloudcommunity.com/ciso-blog-77/securing-the-future-of-agentic-ai-governance-cybersecurity-and-privacy-considerations-3992
  7. https://www.riskinsight-wavestone.com/en/2025/07/agentic-ai-typology-of-risks-and-security-measures/
  8. https://sendbird.com/blog/agentic-ai-challenges
  9. https://www.salesforce.com/blog/unified-trust-security-governance-for-agentic-solutions/
  10. https://www.aicerts.ai/news/navigating-agentic-ai-security-concerns-in-2025-enterprises/
  11. https://www.getmaxim.ai/articles/the-future-of-ai-agents-solving-scalability-challenges-in-enterprise-environments/
  12. https://www.obsidiansecurity.com/blog/agentic-ai-security
  13. https://www.scworld.com/feature/ai-to-change-enterprise-security-and-business-operations-in-2025
  14. https://sintra.ai/blog/autonomous-ai-agents-how-they-work-use-cases-and-sintra-ais-role-in-automation
  15. https://www.isaca.org/resources/news-and-trends/industry-news/2025/safeguarding-the-enterprise-ai-evolution-best-practices-for-agentic-ai-workflows
  16. https://www.mckinsey.com/capabilities/risk-and-resilience/our-insights/deploying-agentic-ai-with-safety-and-security-a-playbook-for-technology-leaders
  17. https://ai-frontiers.org/articles/the-challenges-of-governing-ai-agents
  18. https://www.devopsdigest.com/building-a-governance-framework-for-agentic-ai-systems

Found this useful? Share it.

Written by

Niall McCarthy

More articles by Niall
Sales

Talk to our sales team

Planning @Human for your organisation? Tell us a little and the right person will get back to you.

  • Plans, support and licensing explained
  • Pilot and rollout options
  • Answers from the team that builds @Human

We reply within one working day.

Your enquiry

A few details so we can prepare.

General enquiries

Get in touch

Press, partnerships, careers or anything else. Send us a note and the right person will reply.

  • Press and media
  • Partnerships and community
  • Careers and everything else

Questions about buying @Human? Talk to our sales team.

Your message

We'll reply by email.

Quote

Ask for a quote

Tell us what you need and we'll prepare a written quote for your organisation.

  • Support plans with an SLA
  • Enterprise Support with a named engineer
  • Build, pilot and rollout services

Quotes are free, and the decision is always yours. Prices exclude VAT.

Quote details

The more we know, the more precise the quote.

Newsletter

News from @Human

Product updates, governance tips and early-access news, about once a month.

  • What's new in @Human
  • Practical EU AI Act and ISO 42001 notes
  • Unsubscribe in one click, any time

Your email stays private with us.

Subscribe

Just your email is enough.

Token savings

Estimate what you could save on tokens

Tell us about one process that runs on AI today. We'll map which steps need a model and which can run as token-free code.

  • Which steps need judgement, and which run as code
  • A rough before-and-after of your token bill
  • Ideas you can use whatever you decide

Estimates are illustrative. Real costs depend on your model, provider and volumes.

Get your estimate

A specialist replies within one working day.

Not-for-profit free build

Apply for a free governed AI application

For charities, NGOs and community groups. We build your first @Human application free of charge.

  • Governed from the first line
  • Cheap to run, protects the people you serve
  • Yours to keep and run

We reply within one working day to arrange a scoping call.

Your application

A few details about your organisation and what you'd build.

Enterprise pilot

Plan your @Human pilot

Start with one AI system that matters, prove the value, then standardise what works across the enterprise.

  • Policies, risk and approval gates set up with you
  • Your cloud, your models, your data
  • A clear path from pilot to portfolio

We reply within one working day to plan next steps.

Tell us about your pilot

Two minutes now saves a meeting later.

Ask us anything

Ask your own question

Security, compliance, integration, cost or anything else. A real person replies.

  • An answer within one working day
  • From the team that builds @Human
  • Routed to the right specialist on our team

Prefer to see it live? Book @Human Walkthrough from any page.

Your question

We'll reply by email.

Get started

Choose your @Human plan

Pick what you'd like to pay. Nothing is charged today: we'll email you to get you set up.

  • €0 due today
  • The same platform at every amount
  • Change or cancel at any time

We email you before anything is charged.

Your plan

Three short steps.

Plan€10 a month
Due today€0.00

€0 due today. Choosing your plan gets you started, and we will email you before anything is charged. You can change or cancel at any time.

20-minute walkthrough

See @Human at work with your use cases

Bring a use case you're planning. We'll show you how to build it in @Human as a compliance-first AI system.

  • Built around your own use case
  • How it's classified, governed and evidenced
  • Which plan fits, honestly

Your pace, your decision. A real conversation.

Book your walkthrough

We reply within one working day to find a time.