[object Object]

The compliance officer wants the model on a tenant the legal team controls. The CFO has an existing Azure commitment to burn down. The architect wants the same model serving both Now Assist and the company’s other AI surfaces for consistency. BYO LLM exists for these legitimate reasons. It also exists as a way to take on operational responsibility that ServiceNow’s default model carried for you — which is not always a trade you want to make.

What BYO LLM Enables

Since the Washington release, customers can use pre-approved LLMs (Azure OpenAI, hosted models, certain self-hosted variants) with the platform’s GenAI layer instead of ServiceNow’s default model. Compliance teams who need specific model residency or version pinning now have a supported path. The integration uses the same Now Assist surfaces — incident summarization, case resolution, knowledge generation — but routes inference through your endpoint.

When It Fits

Regulated industries requiring specific model residency (financial services in the EU, healthcare in jurisdictions with strict data localization). Organizations with strong existing LLM investments such as Azure OpenAI enterprise commitments, bespoke fine-tunes on proprietary data, or self-hosted inference at scale. Cost-sensitive deployments where self-hosted inference is genuinely cheaper than the platform’s default at expected volume — verify the math before committing; the operational cost is real.

Strong fit signals:
  - Existing enterprise LLM contract with unused capacity
  - Regulatory requirement for specific model or region
  - Mature MLOps team to operate the endpoint
  - Volume high enough to amortize the operational cost

Governance

BYO LLM does not automatically inherit all Now Assist protections. Configure PII masking, prompt logging, response logging, and audit trails explicitly for your endpoint. The responsibility shifts to you — ServiceNow’s default model covers this automatically through its built-in governance plane. Treat the BYO endpoint as a first-class system with the same audit and access controls you would apply to any production data system.

Setup Path

Register the LLM endpoint in Now Assist admin with URL, authentication, model name, and capability declaration. Configure authentication via the platform credential store. Test with non-sensitive queries first to validate routing and response format. Monitor latency, error rates, and cost per call versus the default. Plan an explicit fallback policy to the default model (or fail-closed) for availability incidents on the BYO endpoint.

// Smoke test pattern
function smokeTestEndpoint(endpointSysId) {
  var endpoint = new GlideRecord('sn_byo_endpoint');
  endpoint.get(endpointSysId);
  var response = callEndpoint(endpoint, {
    prompt: 'Summarize: test message for BYO smoke test',
    max_tokens: 50
  });
  return response.success && response.latency_ms < 5000;
}

Cost Considerations

Run the cost math against expected production volume, not pilot volume. The default ServiceNow model carries a clear per-feature cost; the BYO endpoint adds the LLM call cost plus the operational cost of running the endpoint (or paying the provider). Self-hosted inference often looks cheaper on paper and lands above the SaaS default once you include the team needed to operate it. Pre-commit pricing on Azure OpenAI shifts the math significantly when the commitment is already paid for.

Common Failure Modes

Endpoint registered but governance configured later (or never) — set up logging, masking, and audit at the same time as the endpoint, before any feature uses it. Fallback path untested until needed — exercise the fallback monthly so it works under load. Model version pinned at registration but the provider deprecates the version — track provider lifecycle announcements and have a tested swap procedure for version changes.

What Changed in 2026

The Zurich release expanded the supported BYO endpoint catalog and added the AI Control Tower as the central governance surface for both BYO and default endpoints. If you have BYO endpoints from older releases, audit them against the Control Tower’s policy framework after upgrade — older configurations may not pass current policy checks.

What to do this week: list every Now Assist feature in use and ask which ones genuinely need BYO; the rest can stay on the default and avoid the operational overhead.

[object Object]
Share