---
title: "Automation failure modes"
description: "Diagnose Alpacon automation failures by HTTP status, error code and command result. Fix token approval, scope, rate, account, ACL and sudo problems in CI jobs."
---

> Full documentation index: https://docs.alpacax.com/llms.txt

# Automation failure modes

Start with the HTTP status and `code`, then check the command result if the request was accepted. Coded API errors contain `code`; validation errors can also contain `field_errors`. Some refusals include `gate` and `missing`, but those fields are not present on every error. See [service tokens](/use/iam/applications/service-tokens/) for setup and [Ansible](/integrate/ansible/) for connection options.

## 401 during login with an old CLI

A CLI older than v1.7.4 can accept an `alpst-` key at login and then fail to verify the user profile. The request returns `401 auth_token_missing`, and the old CLI prints:

```text
Login succeeded but failed to verify user profile: request denied by server. Please try logging in again.
```

Use Alpacon CLI v1.7.4 or later and check the version inside the job's container or runner. Pass the key with `alpacon login ... -t "$ALPACON_SERVICE_TOKEN"`. The CLI does not read `ALPACON_URL` or `ALPACON_TOKEN`; those environment variables configure the Ansible connection.

## 202 while a token waits for approval

Creating a service token returns `202` when approval is required. The response includes `key`, `status: "pending_approval"`, `risk_level`, `risk_details` (entries with `scope`, `reason`, and `level`), and `approval_request_id`. A key returned here is not ready to use: it returns `401 auth_authentication_failed` until approval.

Only low-risk scopes requested in the web console by an account with the admin role activate immediately. Command and file-transfer scopes always wait. Have a Superuser or an approver for the request approve it in the web console. Check `GET /api/auth/service-tokens/tokens/{id}/` for `activation_state: "active"` and `enabled: true` before starting the job.

A security change also returns `202` with `status`, `risk_level`, `risk_details`, and `approval_request_id`, but no key. The old settings remain in force until approval. A second pending change returns `400 service_token_update_pending`; finish or cancel the existing request first.

A command waiting for a person has `status: "awaiting_approval"` on its command record; this is different from a token-creation `202`. Review the existing command in the web console.

## 403 with missing permissions

A token missing a scope can receive:

```json
{
  "code": "api_token_scope_missing",
  "gate": "token_scope",
  "missing": "scope:command:create"
}
```

`missing` can be one string or a list. It can name scopes or requirements such as `role`, `command_preapproval`, `approval`, `presence`, `work_session`, `plan`, `approved_account`, `other_approver`, or `allowed_agent`. For the example above, the CLI prints `permission denied: missing scope scope:command:create`.

Check that both the application's global role and the token allow the operation. Request the needed token-scope change and wait for approval. A role refusal uses `rbac_permission_required` or `rbac_object_permission_required`, `gate: "role"`, and `missing: "role"`; the CLI prints `permission denied: missing role`. Ask an administrator to review the role assignment. Adding a scope does not satisfy an account, approval, or presence requirement.

## 429 and Retry-After

A service token over its request limit receives `429` with:

```json
{"code": "api_rate_limited", "gate": "rate"}
```

The `Retry-After` header gives the number of seconds to wait. Limits apply per token; the default is `1000/hour` when no rate is configured. Wait at least that long before retrying and reduce concurrent polling or job requests. A change to the token's rate limit needs approval.

The CLI prints `too many requests—please try again later` and backs off in its polling loops. Ansible raises the HTTP 429 error immediately, so arrange the retry in the job. A monthly plan cap returns `402`, not `429`, and also includes `Retry-After`.

## Sudo refused

On Alpacon Cloud, service-token sudo is allowed below Critical, including unassessed commands, and refused at Critical unless the workspace is Advisory. In Advisory, it is not refused for risk at any level, and the decision is recorded. Outside Advisory, a workspace `token_exec` rule can tighten the risk threshold. The workspace sudo setting must be on; otherwise sudo is refused with `workspace_sudo_with_mfa_disabled`. A risk refusal uses `sudo_risk_denied`. If the whole command is rejected before delivery, its status is `rejected`; read `rejection_cause`, `rejection_gate`, and `rejected_at` on command detail. Ansible returns `rc: 1` with `rejected` on stderr. For a command that ran but failed at sudo, read its output and the **Denied** tab of [Privilege elevation](/use/audit/events/sudo/).

The CLI's risk-denial hint is `sudo was denied by runtime risk assessment; this command is not permitted in this work session.` It uses that wording for token commands too. Review the refused operation and use a narrower command that the application is allowed to run.

On a default self-hosted installation, every service-token sudo is refused with `sudo_command_not_authorized`, regardless of risk. The CLI explains that the service token cannot be elevated on that deployment. Ansible uses only tokens, so `become` is unavailable on a default self-hosted installation. Sudo policies do not apply to service tokens, and retrying does not create a human sudo-approval request.

## 401 after a token expires or a key changes

An expired service token returns `401` with `{"code": "auth_authentication_failed"}` and `WWW-Authenticate: Token`. The same response covers invalid, disabled, pending, and deleted tokens, including tokens whose application was deleted. The response alone cannot distinguish them. The CLI prints `authentication failed: please run 'alpacon login' again`.

Check the token's status and expiry in the web console. For an expired token, issue a replacement, complete its approval, and update the CI secret before retrying. If a key was rotated, replace the CI secret immediately: the old key stops as soon as rotation completes. Rotation resets expiry from the current time, using the rotation policy (90 days by default), so check the new date too. If the token was disabled, re-enabling needs approval; if it is pending, finish approval before retrying.

## IP address or access time refused

`401 auth_ip_not_allowed` means the runner's address is outside the token's allowed IPs. `401 auth_access_outside_allowed_time` means the request is outside the permitted hours or days, evaluated in UTC. The CLI prints `request denied by server` for these coded errors.

Check the runner's outgoing IP and schedule against the token's restrictions. Request a restriction change if the job needs different access; that change requires approval.

## Credential verification unavailable

`503 auth_verification_unavailable` means the server could not verify the credential, rather than rejecting it. The CLI prints `the server could not verify your credential right now—it was not rejected, so try again in a moment`. Retry after a short delay.

## 400 for the server account

A service token can run only as an account owned by its application. Check the validation code and `field_errors`:

| Code | Cause and action |
| --- | --- |
| `workspace_direct_root_disabled` | `root` was requested while direct root is disabled. Use the application's account. |
| `application_system_user_not_found` | No bound account or provisionable application username was found. Bind the account, or set the application's username and enable provisioning on a supported Linux server with a current agent. |
| `system_user_bound_under_another_name` | The bound account uses another name. Use the bound account's name. |
| `server_username_groupname_invalid` | The selected account does not belong to the application. Choose an application-owned account. |
| `system_user_access_denied` | The account's login is disabled or expired. Ask its administrator to review its status. |

For Ansible, set `ansible_user` explicitly; its default of `root` cannot be used with a service token. ACL rules cannot grant ownership of another account.

## Server not connected or commissioned

A request for one `server` returns `400 server_not_connected` or `400 server_not_commissioned` when the target is unavailable for commands. Restore the connection or commission the server before retrying. A request using `servers` can queue for disconnected targets, so check each command's status instead of assuming that acceptance means execution.

## Token scopes exceed the application's role

Token creation returns `400 service_token_scope_exceeds_role` when the requested scopes exceed the application's global role. Have an authorized administrator assign the required global role, or reduce the scopes before requesting the token again. If a role is revoked later, tokens that exceed the remaining permissions are disabled; their next request returns `401`.

## Server, command, or file ACL refused

`403 api_token_acl_not_allowed` means an access rule does not allow the operation. It has no `missing` field. The CLI prints `denied by token access control—this token may not perform that action; review its rules with 'alpacon token acl'`.

For a service token, a single-server request can instead return `400 does_not_exist` under `field_errors.server` when the server is outside its server ACL or does not exist. Check the server ID and add the server rule after the token is approved. For file transfers, allow the path and direction: upload, download, or both.

On Cloud, a command outside the pre-approved commands is accepted with `201`, then ends `rejected` with `rejection_cause` `command_acl_denied` or `command_acl_unavailable`. Ansible reports `Command was rejected by the approval policy` for this too; it can mean a missing command rule. On self-hosted deployments, the command ACL refusal is `403 api_token_acl_not_allowed`; check the ACL against the entire submitted line. On Cloud, ask a **Command ACL admins** member to manage **Pre-approved commands** on the service token's page in the web console. API command-ACL writes return `405 command_acl_moved` there; repeating the CLI hint's write cannot fix it.

## Inline credentials refused

`400 command_inline_credential` means the submitted command contains an inline credential. Remove the credential from the command line and supply it through the job's secret handling before retrying.

## Interactive operation refused

`403 auth_interactive_channel_requires_presence` means a token was used for an interactive operation such as Websh, a tunnel, or the browser editor. Use `alpacon exec` for a non-interactive command, or sign in with `alpacon login` without `-t` for interactive work.

## Workspace suspended

Writes return `403 workspace_suspended` while the workspace is suspended. Ask a workspace administrator to resolve the suspension before retrying the job.

## Ansible configuration or file transfer fails

Missing `ALPACON_URL` or `ALPACON_TOKEN` produces an Ansible required-option error. Set the workspace URL and approved `alpst-` key. An ambiguous server name produces `Multiple servers match name ...`; set `alpacon_server_id` to the intended server UUID. File transfers require S3-backed storage; confirm that storage and the token's file ACLs before retrying. See [Ansible](/integrate/ansible/) for the complete connection options.

Source: https://docs.alpacax.com/integrate/automation-failure-modes/