Shipeasy
APIOperations

Submit a review agent's eval

Alias. Kept for the v1 item routes: it resolves the item's trail and behaves exactly like POST /api/admin/trails/{handle}/evals, answering 404 when the item…

Alias. Kept for the v1 item routes: it resolves the item's trail and behaves exactly like POST /api/admin/trails/{handle}/evals, answering 404 when the item is on no trail. Prefer the /api/admin/trails routes. Not part of the CLI or MCP surface.

Record one review agent's result on one push of a change: a value (0–100 for a score agent, 1/0 for a boolean one), a short rationale, and any findings. threshold and polarity. Submitting again for the same push replaces the earlier result and the open findings it raised.

The agentKey normally names a configured review agent; an unconfigured key is recorded too, as a non-blocking score.

Use case

A review run fired on a push scores the diff for risk and submits { agentKey: "risk", value: 35, rationale: "...", findings: [...] }.

POST
/api/admin/ops/{handle}/trail/evals

Authorization

bearerSdkKey
AuthorizationBearer <token>

Pass an admin SDK key as Authorization: Bearer sdk_admin_…. Mint via POST /api/admin/keys with type: "admin".

In: header

Path Parameters

handle*string

Per-project item number (e.g. 7) or the full ops item id.

Length1 <= length <= 128

Header Parameters

X-Project-Id?string

Project the request operates on. Optional — defaults to the project the SDK key belongs to; pass it only to scope a multi-project key (the generated client sets it once from its configuration, so per-call callers never thread it).

Request Body

application/json

TypeScript Definitions

Use the request body type in TypeScript.

Response Body

application/json

application/json

application/json

application/json

application/json

application/json

curl -X POST "https://shipeasy.ai/api/admin/ops/string/trail/evals" \  -H "Content-Type: application/json" \  -d '{    "agentKey": "risk",    "value": 35,    "rationale": "Touches the token refresh path; covered by the existing auth tests",    "findings": [      {        "severity": "medium",        "title": "Refresh failure is swallowed",        "path": "src/auth/refresh.ts",        "line": 42      }    ]  }'
{
  "eval": {
    "id": "string",
    "agentKey": "string",
    "headSha": "string",
    "status": "pending",
    "value": 0,
    "rationale": "string",
    "reviewer": "string",
    "sessionUrl": "string",
    "error": "string",
    "createdAt": "string",
    "completedAt": "string"
  },
  "findings": [
    {
      "id": "string",
      "agentKey": "string",
      "headSha": "string",
      "severity": "high",
      "title": "string",
      "body": "string",
      "path": "string",
      "line": 0,
      "suggestion": "string",
      "status": "open",
      "stale": true,
      "statusReason": "string",
      "statusBy": "string",
      "promotedItem": {
        "id": "string",
        "number": 0,
        "type": "bug",
        "title": "string",
        "status": "open"
      },
      "createdAt": "string",
      "updatedAt": "string"
    }
  ]
}
{
  "error": "Malformed request (bad JSON, missing project scope).",
  "code": "BAD_REQUEST"
}
{
  "error": "Missing or invalid admin SDK key.",
  "code": "UNAUTHORIZED"
}

{
  "error": "Key is valid but not allowed to act on this project.",
  "code": "FORBIDDEN"
}

{
  "error": "The resource does not exist or is not visible to the caller.",
  "code": "NOT_FOUND"
}

{
  "error": "The request body failed structural (schema) validation.",
  "code": "VALIDATION"
}

Was this page helpful?