Skip to main content

Self-Serve Deployments

Self-serve deployments provide dedicated inference endpoints with reserved capacity on Crusoe's optimized inference engine and managed infrastructure. They give you predictable performance, dedicated throughput, and scalability that grows with your workload.

When to use self-serve deployments​

Self-serve deployments are best when you need:

  • Predictability — Consistent, predictable inference performance over time
  • Sustained traffic — High-volume processing that benefits from reduced cost-per-token as you scale
  • Fine-tuned models — Deployment of your own Low-Rank Adaptation (LoRA) adapters trained through Serverless Fine-Tuning offering
  • Control over scaling — Reduced risk of rate limiting and direct control over the number of replicas supporting your workload
  • Pay-per-use flexibility — GPU-hour billing that lets you align cost with actual usage

Self-serve vs. serverless​

If your workload is sporadic or low-volume, serverless might be a better fit. Use the following table to compare the two options across the dimensions that most affect cost and performance.

DimensionSelf-serveServerless
CapacityDedicated deployment with reserved GPUsShared multi-tenant pool
Rate limitsNone; throughput bounded by replica countShared pool limits
BillingGPU-hour (time-based)Per-token
Model supportBase models and LoRA adapters from Serverless Fine-Tuning or Hugging FaceBase models only; no custom or fine-tuned models
Cost efficiencyBest at sustained high utilizationBest for sporadic or low-volume use

Choose an optimization profile​

Every self-serve deployment is configured with one optimization profile per model. The profile determines how the inference engine is tuned for your workload.

ProfileOptimizationBest for
ResponsivenessLow latency, optimized for quick responsesInteractive applications, real-time inference, latency-sensitive workloads
ThroughputCost efficiency at scale, optimized for token volumeBatch processing, high-volume workflows, cost-per-token minimization
BalancedHybrid blend of throughput and responsiveness, optimized to support moderate token volume and latencyGeneral purpose production traffic

Supported models​

Refer to Available models for the full list of supported base models and Multi-LoRA (multiple adapters hot-loaded at runtime) support. See Deployment types to learn when to use each option.

FAQs​

What serving precision does a self-serve deployment use?​

Precision is set by the configuration you choose when creating the deployment and stays fixed for that deployment's lifetime.

Do Self-serve deployments have rate limits?​

Self-serve deployments have no shared multi-tenant rate limits. Maximum throughput is bounded strictly by the number of dedicated GPU replicas you allocate to the endpoint. In contrast, Serverless Inference's rate limits mean all users of the shared pool compete for capacity. You can scale replica count up or down to meet your workload demand and avoid any hard throughput ceiling imposed by a shared service.

How is traffic routed to my Self-serve deployment?​

Self-serve deployments traffic may be served from any Crusoe Cloud location based on current capacity and availability.

Next steps​