AI providers and models
← Back to Index | Previous: Enterprise roles and access
This chapter expands §8.5 AI models and keys. It documents shipped behaviour on the Enterprise workspace page only — not Control’s scaffold Configuration → AI Models & Keys entry.
13.1 Overview
| Item | Detail |
|---|---|
| Where | https://<dai-host>/dashboard/enterprise/ai-models — Enterprise → AI providers & models |
| Who | Enterprise deployment and platform super admin (others receive 404) |
| Tabs | Provider connections (deployment-wide) · Models (catalog + chat picker) · Team keys (BYOK) (per-team OpenAI / Anthropic / Google) |
Provider connection vs team key
| Path | Use for | Enterprise default |
|---|---|---|
| Provider connection | Bedrock, Azure OpenAI, Vertex AI, Custom gateway | Canonical for platform operators configuring models before chat |
| Team key (BYOK) | Per-team OpenAI, Anthropic, Google keys | Secondary — team admins manage their own keys; Bedrock is not a team key |
If an operator tries to add Bedrock under Team keys, the product rejects it and directs them to Provider connections.
How a model reaches chat: connection → catalog row on Models → Sync now (registers in the LLM proxy) → Test (one real completion) → row appears in the chat picker.
13.2 Choosing a provider
| Kind | Credential | Discover on Models tab | Upstream prefix (LiteLLM) |
|---|---|---|---|
| Custom gateway | Gateway API key | Yes — GET {api_base}/models | openai/<id> added on sync (do not type openai/ yourself when adding manually) |
| Bedrock | IAM keys or host role | No — Add model or Bulk import | bedrock/… or bedrock_mantle/… (enforced at save) |
| Azure OpenAI | API key + API version | No — Add model or Bulk import | azure/<deployment> (hint only — wrong value fails at Test) |
| Vertex AI | Service-account JSON | No — Add model or Bulk import | vertex_ai/<model> (hint only) |
OpenAI, Anthropic, and Gemini without a customer gateway are team keys, not provider connections.
13.3 Bedrock
13.3.1 Connect on Provider connections
- Add connection → Bedrock
- Set Region to match where models are enabled in AWS
- Provide IAM access keys or choose host/environment credentials when LiteLLM runs on AWS compute with an instance profile
- Test connection checks field shape and decrypts stored credentials — it does not call AWS Bedrock. A green connection tick is not proof of invoke access; use Test on each model row.
Minimum IAM for invoke (region-scoped ARNs on foundation models):
sts:GetCallerIdentitybedrock:InvokeModelbedrock:InvokeModelWithResponseStream(required for streaming chat)
13.3.2 Model access — commercial vs GovCloud
Listing ≠ entitlement. A model can appear in a catalog or Discover list and still fail chat until AWS account access is granted.
| Partition | How to enable model access |
|---|---|
| Commercial AWS | Grant Marketplace permissions (aws-marketplace:Subscribe, aws-marketplace:ViewSubscriptions). Bedrock auto-enables many models on first invoke (allow ~15 minutes). Anthropic models may need a one-time first-use form. Do not use the Bedrock console Model access page — AWS documents it as GovCloud-focused. |
| AWS GovCloud (US) | Bedrock console → Model access — enable each model, per region |
IAM invoke permission and model entitlement are separate grants; you need both.
13.3.3 Register models
Discover is not available for Bedrock in the current release. Use Add model or Bulk import:
| Field | Value |
|---|---|
| Model ID | Deslicer catalog slug (e.g. bedrock-claude-4-6-sonnet) or a typed Bedrock / inference-profile id |
| Upstream model | Provider-prefixed Bedrock id (e.g. bedrock/anthropic.claude-sonnet-4-6-v1:0). Known slugs auto-fill upstream when left blank |
Save rejects a catalog slug placed only in Upstream — the slug belongs in Model ID.
13.4 Custom gateway
13.4.1 API base URL must include /v1
On Provider connections → Custom gateway, the API base field is pass-through (only a trailing slash is stripped). Include the /v1 suffix yourself:
| You enter | Discover calls | Result |
|---|---|---|
https://llm-gateway.example.com | GET …/models | Usually 404 on OpenAI-compatible gateways |
https://llm-gateway.example.com/v1 | GET …/v1/models | Correct |
Test connection uses the same base and succeeds only when {api_base}/models responds.
13.4.2 Manual model add
Prefer Discover. When adding by hand, copy the exact id from GET {gateway}/v1/models into both Model ID and Upstream model. Deslicer adds the LiteLLM transport prefix on Sync now — do not type openai/ yourself.
| Gateway lists | Type in both fields |
|---|---|
gpt-5-nano | gpt-5-nano |
openai/gpt-5-nano (nested LiteLLM proxy) | openai/gpt-5-nano |
After a Deslicer AI update that changes custom-gateway prefix handling, run Sync now on affected rows so the proxy receives corrected upstream ids.
13.5 Upstream prefixes (reference)
| Connection kind | Upstream form | Enforced at save? |
|---|---|---|
| Bedrock | bedrock/… or bedrock_mantle/… | Yes |
| Custom gateway | Gateway id; sync adds openai/ when needed | Rewritten on sync |
| Azure OpenAI | azure/<deployment> | No — fails at Test if wrong |
| Vertex AI | vertex_ai/<model> | No — fails at Test if wrong |
Cross-region inference profiles use ids such as bedrock/eu.anthropic.claude-…. A bare foundation-model id can fail even when IAM and entitlement look correct — use the profile id AWS documents for your region when Test returns validation or not-found errors.
13.6 Publish and verify
| Action | What it proves |
|---|---|
| Test connection (connection card) | Gateway: live GET …/models. Other kinds: credential shape only |
| Sync now | Model definitions are registered in the LLM proxy — no completion |
| Test (per model row) | One real chat completion — only step that proves invokability |
| Chat picker | Model is enabled, synced, and allowed for the team’s plan tier |
Order that works: Test connection → register models → Sync now → Test each row → confirm in chat.
13.7 Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| Bedrock save error: catalog slug in Upstream | Slug in wrong field | Put slug in Model ID; upstream must be bedrock/… |
| Connection green, every model Test fails | Wrong region, missing entitlement, or wrong upstream id | Match AWS region; fix Marketplace / GovCloud model access; try inference-profile id |
| Model synced, chat fails | Sync ≠ verify | Run Test on the row; read AWS error name (IAM vs entitlement vs profile) |
| Discover 404, connection test passed | api_base missing /v1 | Set https://…/v1, re-test, Discover again |
| Listed or synced model fails in chat | Listed ≠ entitled, or inference profile required | Commercial: Marketplace permissions; GovCloud: Model access page; retry with profile id |
| Custom gateway double-prefix errors | Friendly Model ID with different Upstream | Use gateway id in both fields; Sync now after product update |
| UI Test fails but cause unclear | Need LiteLLM-side /model/info + completion probe | From /opt/deslicer/ai: bash scripts/enterprise/packaged/diagnose-litellm-model.sh <model_id> (§10.20) |
For host-level LLM proxy diagnostics, run the support scripts in Chapter 10 §10.20 before contacting Deslicer Support (§10.21).
LiteLLM fallback noise at install
Enterprise packaged LiteLLM may log fallback attempts into providers with no configured platform keys. That is a known configuration topic (#1583) — it does not mean your provider connection failed. Operator-managed models on this page are independent of those default fallback edges.
13.8 Related chapters
- Post-install pointer: Chapter 8 §8.5
- General install failures: Chapter 10