Skip to content

Support command-based credentials for custom inference endpoints #3

Description

@srgustafson8

Summary

Cosine supports a custom inference.base_url and the --api-key flag.
Cosine does not support a command that supplies short-lived inference credentials (i.e. a custom credential helper).
Enterprise gateways (such as LiteLLM) can issue JWT access tokens that expire after one hour - this is preferable for us to prevent long lived tokens. They also use a refresh token that must remain in secure storage.

Current behaviour

I tested Cosine CLI 2.0.27 with a local OpenAI-compatible endpoint.
The --api-key value became an Authorization: Bearer header.
This method has three problems:

  • The token appears in the child process arguments.
  • The CLI cannot refresh the token during a long session.
  • The CLI cannot request a new token after an HTTP 401 response.

OPENAI_API_KEY did not start an unauthenticated Cosine session during this test.

Requested change

Add a command-based credential source for custom inference endpoints.

For example:

[inference.auth]
command = "ai-gateway-credentials"
args = ["token", "--no-login"]
refresh_interval_seconds = 240

Cosine should run the command without a shell.
The command should write one token to stdout. Cosine should send diagnostics from stderr to its debug log.

Suggested acceptance criteria

  • Cosine does not store the returned token in a configuration file.
  • Cosine does not add the token to process arguments.
  • Cosine refreshes the token after the configured interval.
  • Cosine refreshes the token and retries once after an HTTP 401 response.
  • Concurrent requests share one refresh operation.
  • Command failures do not expose stdout, headers, or response bodies.
  • The feature works in interactive, headless, plan, and swarm modes.
  • The documentation includes a short-lived OAuth token example.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions