> ## Documentation Index
> Fetch the complete documentation index at: https://arize-ax.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Object Detection

> How to declare your model schema and log data for object detection models

### Object Detection Model Overview

Object detection models identify and locate objects within images or videos by assigning them specific bounding boxes.

Applicable Metrics: Accuracy, Euclidian Distance (embeddings)

Click [here](/ax/machine-learning/machine-learning/how-to-ml/monitors/choosing-your-metrics#valid-model-type-and-metric-family-combinations) for all valid model types and metric combinations.

### Object Detection Code Example

<Tabs>
  <Tab title="Python Batch">
    **Example Row**

    | image\_vector                                                  | image\_link                  | prediction\_bboxes                                  | actual\_bboxes | prediction\_categories  | actual\_categories          | actual\_super\_categories | prediction\_scores | Timestamp |
    | -------------------------------------------------------------- | ---------------------------- | --------------------------------------------------- | -------------- | ----------------------- | --------------------------- | ------------------------- | ------------------ | --------- |
    | `[0.24713118374347687, 0.7061651349067688, 1.12...`            | \`\`\`javascript             |                                                     |                |                         |                             |                           |                    |           |
    | "[https://link-to-my-image.png](https://link-to-my-image.png)" |                              |                                                     |                |                         |                             |                           |                    |           |
    | \`\`\`                                                         | `[[50.43, 109.49, 538.21...` | `[[55.39, 107.72, 539.25, 362.9], [554.41, 194....` | `[bus]`        | `[bus, person, person]` | `[vehicle, person, person]` | `[0.9997552]`             | \`\`\`             |           |
    | 1618590882                                                     |                              |                                                     |                |                         |                             |                           |                    |           |

    ````| theme={null}

    ```python
    embedding_feature_column_names={
        "image_embedding": EmbeddingColumnNames(
            vector_column_name="image_vector",
            link_to_data_column_name="url"
        )
    }
    object_detection_prediction_column_names=ObjectDetectionColumnNames(
        bounding_boxes_coordinates_column_name="prediction_bboxes",
        categories_column_name="prediction_categories",
        scores_column_name="prediction_scores"
    )
    object_detection_actual_column_names=ObjectDetectionColumnNames(
        bounding_boxes_coordinates_column_name="actual_bboxes",
        categories_column_name="actual_categories",
    )

    # feature & tag columns can be optionally defined with typing:
    tag_columns = TypedColumns(
        inferred=["name"],
        to_int=["zip_code", "age"]
    )

    # Defina the Schema, including embedding information
    schema = Schema(
        prediction_id_column_name="prediction_id",
        timestamp_column_name="prediction_ts",
        tag_column_names=tag_columns,
        embedding_feature_column_names=embedding_feature_column_names,
        object_detection_prediction_column_names=object_detection_prediction_column_names,
        object_detection_actual_column_names=object_detection_actual_column_names,
    )

    response = arize_client.log(
        model_id= "CV-object-detection",
        model_version= "v1",
        model_type=ModelTypes.OBJECT_DETECTION,
        environment=Environments.PRODUCTION,
        dataframe=df,
        schema=schema
    )
    ````

    <Card horizontal icon="https://storage.googleapis.com/arize-phoenix-assets/assets/images/phoenix-docs-images/gc.ico" href="https://colab.research.google.com/github/Arize-ai/tutorials_python/blob/main/Arize_Tutorials/Embeddings/CV/Arize_Tutorial_CV_Object_Detection_Quality_Drift.ipynb">
      Google Colaboratory
    </Card>

    For more details on Python Batch API Reference, visit here:

    <Card title="Python Batch API Reference" href="/api-clients/python/version-7/ml-logging/arize-pandas" />

    #### Object Detection Prediction & Actual Values

    Arize supports logging object detection prediction and actual values using the `ObjectDetectionColumnNames` object, which can be assigned to the prediction/actual schema parameters, `object_detection_prediction_column_names` and `object_detection_actual_column_names`.

    Object prediction or actual declaration is required to use the object detection model type in Arize.

    ```python theme={null}
    class ObjectDetectionColumnNames(
        bounding_boxes_coordinates_column_name: str
        categories_column_name: str
        scores_column_name: Optional[str] = None # actual ground truth labels wont have scores
    )
    ```

    #### Embedding Features

    In addition to object detection prediction and actual values, Arize supports logging the embedding features associated with the images in an object detection model using the [`EmbeddingColumnNames`](/api-clients/python/version-7/ml-logging/arize-pandas/embeddingcolumnnames) object.

    * The `vector_column_name` should be the name of the column where the embedding vectors are stored. The embedding vector is the dense vector representation of the unstructured input. ⚠️ **Note:** embedding features are not sparse vectors.

    * The `link_to_data_column_name` should be the name of the column where the URL links to the source images are stored.

    ```javascript theme={null}
    { 
        "embedding_display_name": EmbeddingColumnNames(
            vector_column_name="image_vector", 
            link_to_data_column_name="image_link" 
        ) 
    }
    ```

    See [here](/ax/machine-learning/computer-vision/how-to-cv/embedding-drift) for more information on embeddings and options for generating them.
  </Tab>
</Tabs>