Salesforce deploys cross-Region inference routing in Amazon SageMaker AI

Executive Briefing
“Salesforce is utilizing Amazon SageMaker AI's new cross-Region inference routing capability...”
Key Takeaways
- 1.Salesforce’s Agentforce Foundation team serves in-house and open-source models for various products across 30+ global regions and hundreds of AWS accounts.
- 2.The inference router acts as a regional resource in an account, allowing applications to invoke a single endpoint and automatically fail over to healthy destinations in other regions or accounts during local issues.
- 3.During failover testing under local saturation, the router began shifting traffic about two minutes after errors appeared, routing roughly 78 percent of probe traffic to the remote region without recording errors.
Salesforce is utilizing Amazon SageMaker AI's new cross-Region inference routing capability to distribute model workloads across over 30 global regions. This system automatically manages capacity shortfalls while maintaining data residency commitments.
Key Developments & Primary Facts
Salesforce’s Agentforce Foundation team serves in-house and open-source models for various products across 30+ global regions and hundreds of AWS accounts.
The inference router acts as a regional resource in an account, allowing applications to invoke a single endpoint and automatically fail over to healthy destinations in other regions or accounts during local issues.
During failover testing under local saturation, the router began shifting traffic about two minutes after errors appeared, routing roughly 78 percent of probe traffic to the remote region without recording errors.
Strategic Analysis & Implications
Moving from a self-built internal gateway router to a managed SageMaker AI service reduces manual operational burdens like writing custom health checks and switching logic. This mechanism completely walls off requests tied to local data residency, ensuring regulatory compliance.
The system responds to actual error states and health signals rather than relying solely on high latency or isolated signals, preventing unnecessary cross-Region shifts when slow responses stem from lengthy prompts rather than infrastructure failure.
Outlook & Factors to Watch
The article does not detail specific performance characteristics or system behavior when maintaining local endpoint saturation for extended durations.







