Part 1: Vision Pipeline

Goal: Get a computer vision pipeline working.

Skills: Connect a machine to Viam, configure components in the Viam UI, configure services in the Viam UI.

Time: ~15 min

Prerequisites

Before starting this tutorial, you need the can inspection simulation running. Follow the Gazebo Simulation Setup Guide to:

  1. Build the Docker image with Gazebo Harmonic
  2. Create a machine in Viam and get credentials
  3. Start the container with your Viam credentials

Once you see “Can Inspection Simulation Running!” in the container logs and your machine shows Live in the Viam app, return here to continue.

1.1 Find Your Machine Part

In the Viam app, make sure the Configure tab for your machine is selected.

Machine page showing the green Live status indicator next to the machine name.

Your machine is online but empty. To configure it, you’ll add components and services to your machine part. A machine part is the compute hardware for your robot. In this tutorial, your machine part is a virtual machine running Linux in the Docker container.

Find inspection-station-1-main in the Configure tab.

1.2 Configure the Camera

You’ll now add the camera as a component.

What's a component?

In Viam, a component is any piece of hardware: cameras, motors, arms, sensors, grippers. You configure components by declaring what they are, and Viam handles the drivers and communication.

The power of Viam’s component model: All cameras expose the same API—USB webcams, Raspberry Pi camera modules, IP cameras, simulated cameras. Your application code uses the same GetImages() method regardless of the underlying hardware. Swap hardware by changing configuration, not code.

Add a camera component

To add the camera component to your machine part:

  1. Click the + button and select Configuration block
  2. Search for gz-camera
  3. Select gz-camera:rgb-camera
  4. Click Add component
  5. Enter inspection-cam for the name
  6. Click Add component
Camera configuration panel showing the inspection-cam component with JSON configuration section and documentation.

Configure the camera

To configure your camera component to work with the camera in the simulation, you need to specify the correct camera ID. Most components require a few configuration parameters.

  1. In the JSON Configuration section, add:

    {
  "id": "/inspection_camera"
}

  2. Click Save in the top right

Camera configuration panel with the JSON configuration set to inspection_camera.

1.3 Test the Camera

Verify the camera is working. Every component in Viam has a built-in test card right in the configuration view.

Open the test panel

  1. You should still be on the Configure tab with your inspection-cam selected
  2. Look for the Test section at the bottom of the camera’s configuration panel
  3. Click Test to expand the camera’s test card

The camera component test card uses the camera API to add an image feed to the Viam app, enabling you to determine whether your camera is working. You should see a live video feed from the simulated camera. This is an overhead view of the conveyor/staging area.

Camera test panel showing a live video feed from the simulated inspection camera.

1.4 Add an ML Model Service

Now you’ll add machine learning to run inference on your camera feed. You’ll configure two services:

  • ML model service—Loads a trained model for the inference task
  • Vision service—Connects the camera to the ML model and returns detections

Create the ML model service

  1. Click + next to your machine part
  2. Select Configuration block
  3. Search for tflite
  4. Select tflight_cpu/tflight_cpu
  5. Click Add component
  6. Name it model-service
  7. Click Add component
ML model service configuration panel showing the model-service with Select model button and TFLite CPU module.

Select a model from the registry

Configure the model-service ML model service you just included in your configuration.

  1. In the model-service configuration panel, click Select model

  2. Search for can-defect-detection and select it from the list (a model that classifies cans as PASS or FAIL based on defect detection)

    Select a model dialog showing the can-defect-detection model by Viam.

  3. Click Choose to save the model selection

  4. Click Save in the upper right corner to save your configuration

1.5 Add a Vision Service

Now add a vision service that connects your camera to the ML model service.

Create the vision service

  1. Click + next to your machine part
  2. Select Configuration block
  3. Search for vision
  4. Select mlmodel
  5. Click Add component
  6. Name it vision-service
  7. Click Add component
Vision service configuration panel showing the newly created vision-service with empty ML Model and Default Camera dropdowns.

Link the camera and model in the vision service

  1. Select the vision-service service in your machine’s configuration
  2. Find the ML Model dropdown and select model-service (the ML model service you just created)
  3. Find the Default Camera dropdown and select inspection-cam
  4. Find the Attributes section and set Minimum confidence threshold to 0.75
  5. Click Save in the upper right corner
Vision service configuration panel showing ML Model set to model-service, Default Camera set to inspection-cam, and confidence threshold at 0.75.

Test the vision service

  1. Find the Test section at the bottom of the vision-service configuration panel
  2. Expand the Test card
  3. If not already selected, select inspection-cam as the camera source
  4. Set Detections/Classifications to Live
  5. Check that detection and labeling are working
Vision service test panel showing a can detected with a bounding box and FAIL label.

Continue to Part 2: Data Capture →

Previous

Your First Project

Next

Part 2: Data Capture