Amazon SageMaker HyperPod Inference Gateway for scalable LLM inference
Amazon SageMaker HyperPod Inference Gateway is a Kubernetes-native, GPU-aware routing system that deploys as a single EKS managed add-on on existing SageMaker HyperPod infrastructure with zero application changes.
AWS New (What's New)1 min read