Object detection with TensorFlow

Object detection is important in synthetic intelligenceserving because the spine for quite a few cutting-edge purposes. From autonomous automobiles and surveillance techniques to medical imaging and augmented actuality, the flexibility to establish and find objects in pictures and movies is remodeling industries world wide. A robust and versatile device, the TensorFlow Object Detection API simplifies the creation of strong object detection fashions. By leveraging this API, builders can practice customized fashions tailor-made to particular wants, considerably decreasing growth time and complexity.

On this information, we’ll discover the step-by-step course of of coaching an object detection mannequin utilizing TensorFlow, specializing in integrating knowledge units from Roboflow Universea wealthy repository of annotated knowledge units designed to speed up AI growth.

Studying goals

• Learn to set up and configure TensorFlowObject detection API surroundings for environment friendly mannequin coaching.
• Perceive the best way to put together and preprocess knowledge units for coaching, utilizing the TFRecord format.
• Acquire expertise deciding on and customizing a pre-trained object detection mannequin for particular wants.
• Learn to tune pipeline configuration information and tune mannequin parameters to optimize efficiency.
• Grasp the coaching course of, together with dealing with checkpoints and evaluating mannequin efficiency throughout coaching.
• Perceive the best way to export the skilled mannequin for inference and deployment in real-world purposes.

This text was revealed as a part of the Information Science Blogathon.

Step-by-step implementation of object detection with TensorFlow

On this part, we’ll stroll you thru a step-by-step implementation of object detection utilizing TensorFlow, guiding you from setup to deployment.

Step 1: Arrange the surroundings

The TensorFlow object detection API requires a number of dependencies. Begin by cloning the TensorFlow mannequin repository:

# Clone the tensorflow fashions repository from GitHub
!pip uninstall Cython -y # Short-term repair for "No module named 'object_detection'" error
!git clone --depth 1 https://github.com/tensorflow/fashions

• Uninstall Cython: This step ensures that there are not any conflicts with the Cython library throughout set up.
• Clone the TensorFlow mannequin repository: This repository accommodates the official TensorFlow fashions, together with the article detection API.

Copy the configuration information and modify the setup.py file

# Copy setup information into fashions/analysis folder
%%bash
cd fashions/analysis/
protoc object_detection/protos/*.proto --python_out=.
#cp object_detection/packages/tf2/setup.py .

# Modify setup.py file to put in the tf-models-official repository focused at TF v2.8.0
import re
with open('/content material/fashions/analysis/object_detection/packages/tf2/setup.py') as f:
    s = f.learn()

with open('/content material/fashions/analysis/setup.py', 'w') as f:
    # Set fine_tune_checkpoint path
    s = re.sub('tf-models-official>=2.5.1',
               'tf-models-official==2.8.0', s)
    f.write(s)

Why is that this crucial?

• Compilation of protocol buffers: The Object Detection API makes use of .proto information to outline mannequin configurations and knowledge constructions. These have to be compiled into Python code to work.
• Dependency Model Compatibility: TensorFlow and its dependencies evolve. Utilizing tf-models-official>=2.5.1 could inadvertently set up an incompatible model for TensorFlow v2.8.0.
• Explicitly setting tf-models-official==2.8.0 avoids potential model conflicts and ensures stability.

Putting in dependency libraries

TensorFlow fashions sometimes rely on particular library variations. Fixing the TensorFlow model ensures a clean integration.

# Set up the Object Detection API 

# Have to do a short lived repair with PyYAML as a result of Colab is not capable of set up PyYAML v5.4.1
!pip set up pyyaml==5.3
!pip set up /content material/fashions/analysis/

# Have to downgrade to TF v2.8.0 because of Colab compatibility bug with TF v2.10 (as of 10/03/22)
!pip set up tensorflow==2.8.0

# Set up CUDA model 11.0 (to keep up compatibility with TF v2.8.0)
!pip set up tensorflow_io==0.23.1
!wget https://developer.obtain.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
!mv cuda-ubuntu1804.pin /and so forth/apt/preferences.d/cuda-repository-pin-600
!wget http://developer.obtain.nvidia.com/compute/cuda/11.0.2/local_installers/cuda-repo-ubuntu1804-11-0-local_11.0.2-450.51.05-1_amd64.deb
!dpkg -i cuda-repo-ubuntu1804-11-0-local_11.0.2-450.51.05-1_amd64.deb
!apt-key add /var/cuda-repo-ubuntu1804-11-0-local/7fa2af80.pub
!apt-get replace && sudo apt-get set up cuda-toolkit-11-0
!export LD_LIBRARY_PATH=/usr/native/cuda-11.0/lib64:$LD_LIBRARY_PATH

Whereas operating this block, you will need to restart classes once more and run this code block once more to efficiently set up all dependencies. It will set up all of the dependencies efficiently.

Putting in an acceptable model of the protobuf library to resolve dependency points

!pip set up protobuf==3.20.1

Step 2: Examine the surroundings and amenities

To verify that the set up works, run the next take a look at:

# Run Mannequin Bulider Take a look at file, simply to confirm every part's working correctly
!python /content material/fashions/analysis/object_detection/builders/model_builder_tf2_test.py

If no errors seem, your configuration is full. Now we now have accomplished the configuration efficiently.

Step 3: Put together coaching knowledge

For this tutorial, we’ll use the “Particular person detection” knowledge set Roboflow Universe. Observe these steps to organize it:

Go to the dataset web page:

Fork the dataset into your workspace to make it accessible for personalization.

Construct a model of the information set to finalize your preprocessing settings, similar to scaling and resizing.

Now, obtain it in TFRecord format, which is a binary format optimized for TensorFlow workflows. TFRecord shops knowledge effectively and permits TensorFlow to learn giant knowledge units throughout coaching with minimal overhead.

As soon as downloaded, place the dataset information in your Google Drive, mount your code to your drive, and add these information to the code to make use of it.

from google.colab import drive
drive.mount('/content material/gdrive')


train_record_fname="/content material/gdrive/MyDrive/pictures/practice/practice.tfrecord"
val_record_fname="/content material/gdrive/MyDrive/pictures/take a look at/take a look at.tfrecord"
label_map_pbtxt_fname="/content material/gdrive/MyDrive/pictures/label_map.pbtxt"

Step 4: Configure Coaching Settings

Now it is time to set the configuration for the article detection mannequin. For this instance, we’ll use the efficientdet-d0 mannequin. You’ll be able to select from different fashions like ssd-mobilenet-v2 or ssd-mobilenet-v2-fpnlite-320, however for this information, we’ll concentrate on efficientdet-d0.

# Change the chosen_model variable to deploy totally different fashions accessible within the TF2 object detection zoo
chosen_model="efficientdet-d0"

MODELS_CONFIG = {
    'ssd-mobilenet-v2': {
        'model_name': 'ssd_mobilenet_v2_320x320_coco17_tpu-8',
        'base_pipeline_file': 'ssd_mobilenet_v2_320x320_coco17_tpu-8.config',
        'pretrained_checkpoint': 'ssd_mobilenet_v2_320x320_coco17_tpu-8.tar.gz',
    },
    'efficientdet-d0': {
        'model_name': 'efficientdet_d0_coco17_tpu-32',
        'base_pipeline_file': 'ssd_efficientdet_d0_512x512_coco17_tpu-8.config',
        'pretrained_checkpoint': 'efficientdet_d0_coco17_tpu-32.tar.gz',
    },
    'ssd-mobilenet-v2-fpnlite-320': {
        'model_name': 'ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8',
        'base_pipeline_file': 'ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.config',
        'pretrained_checkpoint': 'ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.tar.gz',
    },
}

model_name = MODELS_CONFIG(chosen_model)('model_name')
pretrained_checkpoint = MODELS_CONFIG(chosen_model)('pretrained_checkpoint')
base_pipeline_file = MODELS_CONFIG(chosen_model)('base_pipeline_file')

Then we obtain the pre-trained weights and the configuration file similar to the chosen mannequin:

# Create "mymodel" folder for holding pre-trained weights and configuration information
%mkdir /content material/fashions/mymodel/
%cd /content material/fashions/mymodel/

# Obtain pre-trained mannequin weights
import tarfile
download_tar="http://obtain.tensorflow.org/fashions/object_detection/tf2/20200711/" + pretrained_checkpoint
!wget {download_tar}
tar = tarfile.open(pretrained_checkpoint)
tar.extractall()
tar.shut()

# Obtain coaching configuration file for mannequin
download_config = 'https://uncooked.githubusercontent.com/tensorflow/fashions/grasp/analysis/object_detection/configs/tf2/' + base_pipeline_file
!wget {download_config}

After this, we configure the variety of steps for coaching and the batch dimension relying on the chosen mannequin:

# Set coaching parameters for the mannequin
num_steps = 4000

if chosen_model == 'efficientdet-d0':
  batch_size = 8
else:
  batch_size = 8

You’ll be able to enhance and reduce num_steps and batch_size as per your necessities.

Step 5: Modify the Pipeline Configuration File

We have to customise the pipeline.config file with the paths to our dataset and mannequin parameters. The pipeline.config file accommodates numerous settings, similar to batch dimension, variety of courses, and fine-tuning checkpoints. We make these modifications by studying the template and changing the related fields:

# Set file places and get variety of courses for config file
pipeline_fname="/content material/fashions/mymodel/" + base_pipeline_file
fine_tune_checkpoint="/content material/fashions/mymodel/" + model_name + '/checkpoint/ckpt-0'

def get_num_classes(pbtxt_fname):
    from object_detection.utils import label_map_util
    label_map = label_map_util.load_labelmap(pbtxt_fname)
    classes = label_map_util.convert_label_map_to_categories(
        label_map, max_num_classes=90, use_display_name=True)
    category_index = label_map_util.create_category_index(classes)
    return len(category_index.keys())
num_classes = get_num_classes(label_map_pbtxt_fname)
print('Complete courses:', num_classes)

# Create customized configuration file by writing the dataset, mannequin checkpoint, and coaching parameters into the bottom pipeline file
import re

%cd /content material/fashions/mymodel
print('writing customized configuration file')

with open(pipeline_fname) as f:
    s = f.learn()
with open('pipeline_file.config', 'w') as f:

    # Set fine_tune_checkpoint path
    s = re.sub('fine_tune_checkpoint: ".*?"',
               'fine_tune_checkpoint: "{}"'.format(fine_tune_checkpoint), s)

    # Set tfrecord information for practice and take a look at datasets
    s = re.sub(
        '(input_path: ".*?)(PATH_TO_BE_CONFIGURED/practice)(.*?")', 'input_path: "{}"'.format(train_record_fname), s)
    s = re.sub(
        '(input_path: ".*?)(PATH_TO_BE_CONFIGURED/val)(.*?")', 'input_path: "{}"'.format(val_record_fname), s)

    # Set label_map_path
    s = re.sub(
        'label_map_path: ".*?"', 'label_map_path: "{}"'.format(label_map_pbtxt_fname), s)

    # Set batch_size
    s = re.sub('batch_size: (0-9)+',
               'batch_size: {}'.format(batch_size), s)

    # Set coaching steps, num_steps
    s = re.sub('num_steps: (0-9)+',
               'num_steps: {}'.format(num_steps), s)

    # Set variety of courses num_classes
    s = re.sub('num_classes: (0-9)+',
               'num_classes: {}'.format(num_classes), s)

    # Change fine-tune checkpoint sort from "classification" to "detection"
    s = re.sub(
        'fine_tune_checkpoint_type: "classification"', 'fine_tune_checkpoint_type: "{}"'.format('detection'), s)

    # If utilizing ssd-mobilenet-v2, scale back studying price (as a result of it is too excessive within the default config file)
    if chosen_model == 'ssd-mobilenet-v2':
      s = re.sub('learning_rate_base: .8',
                 'learning_rate_base: .08', s)

      s = re.sub('warmup_learning_rate: 0.13333',
                 'warmup_learning_rate: .026666', s)

    # If utilizing efficientdet-d0, use fixed_shape_resizer as a substitute of keep_aspect_ratio_resizer (as a result of it is not supported by TFLite)
    if chosen_model == 'efficientdet-d0':
      s = re.sub('keep_aspect_ratio_resizer', 'fixed_shape_resizer', s)
      s = re.sub('pad_to_max_dimension: true', '', s)
      s = re.sub('min_dimension', 'top', s)
      s = re.sub('max_dimension', 'width', s)

    f.write(s)


# (Non-obligatory) Show the customized configuration file's contents
!cat /content material/fashions/mymodel/pipeline_file.config

# Set the trail to the customized config file and the listing to retailer coaching checkpoints in
pipeline_file="/content material/fashions/mymodel/pipeline_file.config"
model_dir="/content material/coaching/"

Step 6: Prepare the mannequin

Now we are able to practice the mannequin utilizing the customized pipeline configuration file. The coaching script will save checkpoints, which you need to use to guage the efficiency of your mannequin:

# Run coaching!
!python /content material/fashions/analysis/object_detection/model_main_tf2.py 
    --pipeline_config_path={pipeline_file} 
    --model_dir={model_dir} 
    --alsologtostderr 
    --num_train_steps={num_steps} 
    --sample_1_of_n_eval_examples=1

Step 7 – Save the skilled mannequin

As soon as coaching is full, we export the skilled mannequin so it may be used for inference. We use the exporter_main_v2.py script to export the mannequin:

!python /content material/fashions/analysis/object_detection/exporter_main_v2.py 
    --input_type image_tensor 
    --pipeline_config_path {pipeline_file} 
    --trained_checkpoint_dir {model_dir} 
    --output_directory /content material/exported_model

Lastly, we compress the exported mannequin into a zipper file for straightforward downloading after which you’ll be able to obtain the zip file containing your skilled mannequin:

import shutil
# Path to the exported mannequin folder
exported_model_path="/content material/exported_model"

# Path the place the zip file shall be saved
zip_file_path="/content material/exported_model.zip"

# Create a zipper file of the exported mannequin folder
shutil.make_archive(zip_file_path.change('.zip', ''), 'zip', exported_model_path)

# Obtain the zip file utilizing Google Colab's file obtain utility
from google.colab import information
information.obtain(zip_file_path)

You need to use these downloaded mannequin information to check them on unseen pictures or in your purposes as per your wants.

You’ll be able to test this: collaboration pocket book for detailed code

Conclusion

In conclusion, this information provides you the information and instruments crucial to coach an object detection mannequin utilizing the TensorFlow Object Detection API, leveraging Roboflow Universe datasets for speedy customization. By following the steps outlined, you’ll be able to successfully put together your knowledge, configure the coaching course of, choose the appropriate mannequin, and tune it to fulfill your particular wants. Moreover, the flexibility to export and deploy your skilled mannequin opens up huge prospects for real-world purposes, whether or not in autonomous automobiles, medical imaging, or surveillance techniques. This workflow permits you to create highly effective, scalable object detection techniques with diminished complexity and sooner deployment time.

Key takeaways

• The TensorFlow Object Detection API supplies a versatile framework for creating customized object detection fashions with pre-trained choices, decreasing growth time and complexity.
• The TFRecord format is important for environment friendly knowledge dealing with, particularly with giant knowledge units in TensorFlow, permitting for quick coaching and minimal overhead.
• Pipeline configuration information are essential for tuning and tuning the mannequin to work along with your particular knowledge set and desired efficiency traits.
• Pretrained fashions like efficientdet-d0 and ssd-mobilenet-v2 present strong beginning factors for coaching customized fashions, and every has particular strengths relying on use case and useful resource constraints.
• The coaching course of includes managing parameters similar to batch dimension, variety of steps, and mannequin checkpoints to make sure that the mannequin learns optimally.
• Exporting the mannequin is important to make use of the skilled object detection mannequin on a real-world mannequin that’s being packaged and prepared for deployment.

Continuously requested questions

The media proven on this article shouldn’t be the property of Analytics Vidhya and is used on the writer’s discretion.

I’m Neha Dwivedi, a knowledge science fanatic, graduated from MIT World Peace College, Pune. I’m keen about knowledge science and the rising traits with it. I am excited to share concepts and study from this neighborhood!