Generative AI in 2025: Separating Signal from Noise

Two years after ChatGPT launched, the enterprise generative AI landscape has matured considerably. The breathless hype has faded. The "AI will replace everyone" predictions look silly. And we're finally seeing what actually works.

Here's what we've learned from deploying generative AI across 23 enterprise clients.

What's Actually Working

Internal Knowledge Search

The killer app for enterprise LLMs isn't customer-facing chatbots—it's helping employees find information.

One of our clients, a 40,000-person financial services firm, deployed an internal search system that:

Reduced time-to-answer for compliance questions from 2 hours to 4 minutes
Cut new employee onboarding research time by 60%
Surfaced relevant precedents that humans missed 30% of the time

The key insight: LLMs excel at synthesis and retrieval over large document corpuses. They're mediocre at generating novel content but excellent at finding and summarizing existing content.

Implementation tip: Start with a narrow domain (legal, HR policies, technical documentation) rather than "search everything." Accuracy matters more than breadth.

Code Assistance (With Guardrails)

Developer productivity tools have delivered measurable ROI:

Metric	Before	After	Impact
Time to first commit (new hires)	3 weeks	1 week	66% faster
Boilerplate code time	40% of dev time	15%	25% reclaimed
Code review iterations	3.2 avg	2.1 avg	34% fewer

But the wins came with strict guardrails:

AI-generated code requires human review before merge
Sensitive codebases (auth, payments) are excluded
Regular audits for security vulnerabilities in AI suggestions

Implementation tip: Measure actual productivity, not just "lines of code generated." We've seen teams generate more code but ship fewer features.

Document Drafting (Not Document Writing)

LLMs are excellent first-draft machines for:

Contract redlines and summaries
Meeting notes and action items
Report templates and outlines
Email responses for common scenarios

They're poor at:

Original thought leadership
Nuanced client communications
Anything requiring current information
Content that needs to be factually perfect

Implementation tip: Position AI as "draft generation" not "content creation." Human review isn't optional—it's the product.

What's Still Vaporware

Autonomous Agents

Despite the demos, we haven't seen a single production deployment of truly autonomous AI agents that:

Make consequential decisions without human approval
Operate reliably across multi-step workflows
Handle edge cases gracefully

The technology isn't there yet. Every "autonomous agent" we've evaluated either:

Requires constant human supervision (not autonomous)
Fails catastrophically on edge cases (not production-ready)
Works only in narrow, controlled scenarios (not general-purpose)

Our advice: Wait 18-24 months. The technology is improving rapidly, but early adopters are paying the innovation tax.

Customer Service Replacement

The promise: AI handles 80% of customer inquiries, humans handle the rest.

The reality: AI handles 80% of simple inquiries that were already handled by FAQ pages and basic automation. Complex issues still need humans, and customers are increasingly frustrated by AI gatekeepers.

We've seen customer satisfaction scores drop at companies that over-indexed on AI customer service. The savings in headcount were offset by increased churn.

Our advice: Use AI to augment human agents (faster information retrieval, suggested responses) rather than replace them.

"Just Add AI" to Existing Products

The most common failure pattern: taking an existing product and bolting on a chatbot or AI feature without rethinking the user experience.

Users don't want to chat with their expense reporting software. They want expense reporting to be faster and less annoying. Sometimes AI helps with that. Often, better UX design helps more.

Our advice: Start with the user problem, not the technology. "How might AI help?" is the wrong question. "What's the user trying to accomplish?" is the right one.

The Build vs. Buy Decision

A year ago, everyone wanted to build custom LLM applications. Today, the calculus has shifted:

Build when:

Your use case requires proprietary data that can't leave your infrastructure
Off-the-shelf solutions don't exist for your specific workflow
AI is a core competitive differentiator (not just efficiency)
You have the ML engineering talent to maintain it

Buy when:

The use case is common (search, summarization, code assistance)
Time-to-value matters more than customization
You lack ML infrastructure and expertise
The vendor has a credible enterprise track record

Most enterprises should buy 80% and build 20%. The build portion should focus on proprietary data and workflows that create competitive advantage.

Cost Realities

LLM costs have dropped 90% in two years, but total cost of ownership is still higher than most projections:

Typical Enterprise LLM Project Costs (Year 1)

API/Compute costs:     $50-200K
Integration engineering: $200-500K
Data preparation:       $100-300K
Security/compliance:    $50-150K
Training & change mgmt: $50-100K
                       ___________
Total:                 $450K-1.25M

The API costs that vendors quote are often 10-20% of the total investment. Plan accordingly.

What's Next

Three trends we're watching:

Smaller, specialized models. The era of "one model to rule them all" is ending. Purpose-built models for specific domains (legal, medical, financial) are outperforming general-purpose LLMs at lower cost.

On-premise deployment. Enterprises are increasingly demanding models that run on their infrastructure. Data privacy concerns and regulatory requirements are driving this shift.

Multimodal workflows. The interesting applications combine text, image, and structured data. Document processing that understands both the text and the layout. Customer service that can see what the user sees.

The Executive Takeaway

Generative AI is real and valuable, but it's not magic. The winners are treating it like any enterprise technology: with clear use cases, measurable ROI, and realistic expectations.

The losers are still chasing demos and hoping for transformation.

Evaluating generative AI for your organization? We offer a 2-week rapid assessment that identifies your highest-value use cases and builds a realistic implementation roadmap. Learn more.