How to build scalable agentic AI

Here are the key takeaways from “How to build scalable agentic AI applications for enterprises” (CIO) by Hari Subramanian:

What is “agentic AI” & why it matters

Agentic AI refers to systems of autonomous agents (or multi-agent systems) that can carry out multi-step, complex workflows rather than just one-off responses.
This shift is significant: instead of humans orchestrating AI calls or pipelines manually, these agents can coordinate themselves, interact with tools or data sources, and adapt.
For enterprises, agentic AI offers the potential to scale automation in business processes, improving efficiency, reducing human overhead, and enabling new capabilities.

Core components in agentic AI systems

The article breaks down typical agentic workflows into four core components that must be integrated:

Prompts
- These define goals or tasks for agents.
- Challenges: versioning, testing, portability across models.
MCP servers / protocols
- “MCP” here refers to the protocol layer that allows agents to connect to external tools, services, APIs, or data sources.
- It supports discovery, authentication, and invocation of enterprise tools.
Models
- The “brains” of the system: LLMs or fine-tuned models responsible for reasoning, planning, or generation.
- Challenges: hosting and scaling, latency, cost, vendor lock-in, reliability.
Agents
- The autonomous units that carry out tasks.
- They may be reactive, deliberative, learning, or fully autonomous.
- Key difficulties: debugging, memory/state management, security, orchestrating sub-agents.

Challenges & tradeoffs

Some of the main obstacles engineering teams face when building agentic AI at scale include:

Prompts: No built-in versioning, limited testability, model portability issues.
Models: Running or fine-tuning large models oneself is complex; latency and cost are nontrivial; dependence on third-party models introduces risks.
Tools / MCP servers: Discovery, hosting, tool proliferation, weak access/policies, lack of standard observability.
Agents: Hard to debug, state/memory management is complicated, security concerns, coupling between frontends/backends/ planners.

Because of these challenges, simply piecing components together in an ad hoc way becomes brittle, costly, and hard to maintain.

Architectural solution: Platform + LLM Gateway

The article promotes a platform-centric architectural approach, with a central LLM gateway (or AI gateway) as the orchestration and control plane. Key ideas are:

Think of the LLM gateway akin to an API gateway — a unified interface between agents / applications and multiple models, tools, services.
The gateway abstracts away model complexity, provides routing, fallback, governance, observability, rate limiting, guardrails, tool integration, etc.
This gives enterprises flexibility: multiple models (on-premise, cloud, open source, proprietary), hybrid deployments, regional routing, etc.
A good gateway also supports sandboxing (for prompt / agent experimentation), canary / staged rollouts, pipeline testing, model upgrades, and more.

Key benefits of the platform / gateway approach:

Unified model access — one API to manage many models.
Routing & fallback — based on latency, cost, availability.
Rate limiting & quotas — per team, per model, per user.
Guardrails / safety — enforcing PII filtering, mitigating toxicity or jailbreaks.
Observability & tracing — logs, metrics, prompt / response tracking.
Tool / agent integration — via MCP, agents can call enterprise systems (Jira, collaboration, internal APIs, etc.).
Agent-to-agent protocols (A2A) — allowing agents to discover and interact with each other.
Deployment flexibility — support for hybrid, on-prem, public cloud.

Strategic recommendations & outlook

The author argues that agentic AI is becoming foundational infrastructure, not a niche experiment. To succeed, it must be treated as a mission-critical system from Day 1, not an afterthought.
Designing a coherent architecture early that includes a gateway and platform layers avoids many pitfalls (e.g. fragmented deployments, scalability challenges, security risks).
Enterprises should anticipate a multi-model future (i.e., use of more than one model provider, open source + hosted) — thus, the gateway / routing layer will be increasingly important.
The gateway becomes a leverage point for cost optimization, resiliency, governance, security, and ease of evolving the system over time.

Source:How to build scalable agentic AI applications for enterprises | CIO https://share.google/lZeIj0Fv3OpvwkJ9o

I could only locate one plausible match: “How to Survive Artificial Intelligence” appears to correspond to the essay “AI Will Change What It Is to Be Human” by Tyler Cowen and Avital Balwit (published in The Free Press) which discusses how humanity adapts psychologically and existentially in an age of advanced AI.

Here are key ideas / takeaways from that essay (and related commentary) plus reflections on how to “survive” (i.e. adapt to) AI’s emergence:

---

Key Ideas from the Essay

1. Identity crisis & human obsolescence anxiety

As AI systems increasingly match or exceed human cognitive abilities, we may face a crisis about what it means to be human.

The authors argue that “demoralization” is a real risk: people could struggle to find purpose when machines can replicate many intellectual tasks.

2. Acceleration is already underway

The essay underscores that AI capability is not a distant future — we are in the middle of it.

Examples: GPT-4 outperforming many human tests, Claude 3.5 making sense of diagrams, etc.

3. Humans must reclaim their distinctiveness

The authors suggest that certain qualities — especially those not reducible to prediction or pattern matching — will remain uniquely human: meaning, purpose, subjectivity, values, aesthetic sense.

AI can simulate knowledge or skill, but it (so far) can’t live or feel in the way humans do.

4. Evolving education, scholarship, and creative life

Traditional outputs (e.g. textbooks, monographs, fact-heavy research) may decline in relevance, because AI can generate or replicate them.

Instead, the emphasis should shift to deeper meanings, interpretive work, questions over answers, fostering agency, and the lived dimension of humanism.

5. A new relationship with tools & the archive

The essay calls AI a new way to converse with the archive: the sum of human knowledge, now accessible, remixable, interactive.

The task becomes: how do we shape, dialogue with, and resist being subsumed by that archive?

---

“Survival” Strategies (i.e. Adapting Amid AI)

From that essay, plus from adjacent thought in essays like How to Survive and Thrive in the AI Apocalypse, one can infer a set of strategies to survive (and even thrive) in this new era:

Strategy What It Means Why It Matters

Own your humanity Cultivate what machines can’t do: subjective experience, moral imagination, emotional resonance, existential reflection. These will remain the zones where humans add irreplaceable value.

Be creative, not just predictive Focus on original synthesis, disruptive thinking, leaps beyond data interpolation. AI is strong at optimization & pattern matching; humans must push boundaries.

Adopt a lifelong experimental mindset Be ready to relearn, pivot, unlearn, and explore new domains. The pace of change will make rigid careers less durable.

Work with AI Use AI as an augmentation tool, not as mere competitor or replacement. Those who learn to co-pilot with AI will outlast those who resist it.

Deepen domain mastery + meta-skills Combine domain expertise with critical thinking, judgment, contextual awareness, ethics, narrative capability. These “power skills” impose higher barriers for AI to replicate.

Explore emergent human spaces Meaning, communities, spiritual life, aesthetic consumption, non-instrumental pursuits. If AI captures most instrumental work, human life will shift more to expressive, relational, existential spheres.

Guard against demoralization Cultivate purpose, mentoring, philosophical rootedness, communities of shared inquiry. Psychological resilience will be as essential as technical skill.

Rajesh Sabbani