> ## Documentation Index
> Fetch the complete documentation index at: https://docs.axilio.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Screenshots: locate

> Find the element a query describes in a screenshot: a point, or an OCR text.

Find the element a natural-language query describes, such as
`"the blue Continue button"`, in a screenshot you supply. A vision-language
model reads the image and answers in one of two ways when `found` is true:

* **A box.** `bbox` is the element's pixel box in the screenshot.
* **A matched text.** When you send `texts`, the OCR results for the same
  screenshot, the model can answer with `matched_text_index` instead. It points
  into your `texts` list, so you reuse that entry's exact box.

Argus does not run OCR for this operation. To ground answers in text, call
[Screenshots: detect](/api-reference/argus/screenshots-detect) first and pass
its text results as `texts` unchanged: detect already returns each text box
normalized to `[0, 1]` of the image, which is the form `texts` takes. Only the
answer's `bbox` is in pixels.

Pass a `model` from [Models: list](/api-reference/argus/list-models), or omit
it to use the default. The call is billed per token on that model, and the
response reports `prompt_tokens`, `completion_tokens`, and `cost_microdollars`.

If you already have the screen's UI elements, for example from a DOM, use
[Elements: locate](/api-reference/argus/elements-locate) instead. It answers
with one of your elements, which is exact where a box is approximate.

Errors use the [problem details](/reference/errors#problem-details) shape.


## OpenAPI

````yaml api-reference/openapi-argus.json POST /api/v1/screenshots:locate
openapi: 3.1.0
info:
  title: Argus
  version: 2.0.0
servers:
  - url: https://argus.axilio.ai
    description: Production
security: []
tags:
  - name: Elements
  - name: Models
  - name: Screenshots
paths:
  /api/v1/screenshots:locate:
    post:
      tags:
        - Screenshots
      summary: Locate In Screenshot
      description: >-
        Find the element a query describes in a screenshot: a point, or an OCR
        text.
      operationId: screenshots_locate
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/LocateRequest'
        required: true
      responses:
        '200':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/LocateResponse'
          description: Successful Response
        '400':
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/Problem'
          description: Invalid input (bad image, unsupported model)
        '401':
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/Problem'
          description: Missing or invalid credential
        '402':
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/Problem'
          description: Insufficient balance
        '422':
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/Problem'
          description: Request validation failed
        '500':
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/Problem'
          description: Internal error (masked)
        '502':
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/Problem'
          description: The model provider failed
      security:
        - AxilioApiKey: []
components:
  schemas:
    LocateRequest:
      description: >-
        Argus POST /screenshots:locate request.


        The caller pre-computes OCR via POST /screenshots:detect (or reuses
        cached

        results keyed by frame hash) and passes the text list here. Argus does

        NOT re-run OCR; the VLM call uses the provided `texts` as grounding

        context.
      properties:
        image:
          description: Base64 encoded image (PNG or JPEG)
          maxLength: 16777216
          title: Image
          type: string
        model:
          anyOf:
            - minLength: 1
              type: string
            - type: 'null'
          description: >-
            VLM model to use; must be one of the models from GET /models. Omit
            to use the server's configured default. The system prompt is fixed
            to the element-locator task.
          title: Model
        query:
          description: Natural-language target description
          minLength: 1
          title: Query
          type: string
        texts:
          description: >-
            Pre-computed OCR text elements. Empty list means Argus skips OCR
            grounding and asks the VLM to locate from the image alone.
          items:
            $ref: '#/components/schemas/TextElementInput'
          title: Texts
          type: array
      required:
        - image
        - query
      title: LocateRequest
      type: object
    LocateResponse:
      description: |-
        Argus POST /screenshots:locate response.

        Exactly one of `matched_text_index` or `bbox` is populated when
        `found` is True. `matched_text_index` indicates the VLM grounded its
        answer to a specific OCR text element (pixel-accurate bbox available
        via the original `texts` list). `bbox` is a free-form VLM bbox.
      properties:
        bbox:
          anyOf:
            - $ref: '#/components/schemas/LocateBBox'
            - type: 'null'
          description: >-
            Pixel-space bbox. None when matched_text_index is set or
            found=False.
        completion_tokens:
          default: 0
          description: Completion (output) tokens the model billed for this call
          minimum: 0
          title: Completion Tokens
          type: integer
        confidence:
          description: Model self-reported confidence
          maximum: 1
          minimum: 0
          title: Confidence
          type: number
        cost_microdollars:
          description: Provider-reported cost in microdollars
          minimum: 0
          title: Cost Microdollars
          type: integer
        found:
          description: Whether the VLM located the target
          title: Found
          type: boolean
        latency_ms:
          description: End-to-end VLM call latency
          minimum: 0
          title: Latency Ms
          type: integer
        matched_text_index:
          anyOf:
            - minimum: 0
              type: integer
            - type: 'null'
          description: Index into the request `texts` list when the answer is OCR-grounded.
          title: Matched Text Index
        model:
          description: Model identifier returned by the provider
          title: Model
          type: string
        prompt_tokens:
          default: 0
          description: Prompt (input) tokens the model billed for this call
          minimum: 0
          title: Prompt Tokens
          type: integer
      required:
        - found
        - confidence
        - model
        - cost_microdollars
        - latency_ms
      title: LocateResponse
      type: object
    Problem:
      description: RFC 9457 problem details, as argus (and the backend) answer every error.
      properties:
        code:
          description: Stable machine-readable category, e.g. validation, not_found
          title: Code
          type: string
        detail:
          description: Human-readable explanation; masked for internal errors
          title: Detail
          type: string
        errors:
          anyOf:
            - items:
                additionalProperties:
                  type: string
                type: object
              type: array
            - type: 'null'
          default: null
          description: Per-field messages on a request validation problem
          title: Errors
        status:
          description: The HTTP status code
          title: Status
          type: integer
        title:
          description: The status's reason phrase
          title: Title
          type: string
        type:
          default: about:blank
          description: Always about:blank
          title: Type
          type: string
      required:
        - title
        - status
        - detail
        - code
      title: Problem
      type: object
    TextElementInput:
      description: OCR-detected text element to seed the VLM's grounding context.
      properties:
        bbox:
          $ref: '#/components/schemas/BoundingBox'
          description: Normalized [0, 1] bbox in the source image
        text:
          description: OCR'd text content
          title: Text
          type: string
      required:
        - text
        - bbox
      title: TextElementInput
      type: object
    LocateBBox:
      description: Pixel-space bbox returned by the VLM for the located element.
      properties:
        height:
          description: Height in pixels
          exclusiveMinimum: 0
          title: Height
          type: integer
        width:
          description: Width in pixels
          exclusiveMinimum: 0
          title: Width
          type: integer
        x:
          description: Top-left x in image pixels
          minimum: 0
          title: X
          type: integer
        'y':
          description: Top-left y in image pixels
          minimum: 0
          title: 'Y'
          type: integer
      required:
        - x
        - 'y'
        - width
        - height
      title: LocateBBox
      type: object
    BoundingBox:
      properties:
        x1:
          description: Top-left x coordinate
          title: X1
          type: number
        x2:
          description: Bottom-right x coordinate
          title: X2
          type: number
        y1:
          description: Top-left y coordinate
          title: Y1
          type: number
        y2:
          description: Bottom-right y coordinate
          title: Y2
          type: number
      required:
        - x1
        - y1
        - x2
        - y2
      title: BoundingBox
      type: object
  securitySchemes:
    AxilioApiKey:
      description: Axilio API key (axl_*).
      in: header
      name: X-Axilio-Api-Key
      type: apiKey

````

This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.