【PaddlePaddle+OpenVINO】PP-HumanSeg部署

AI Studio

561人浏览 · 2022-04-08 15:56:48

AI Studio · 2022-04-08 15:56:48 发布

转载自AI Studio 项目链接：https://aistudio.baidu.com/aistudio/projectdetail/3728571?channelType=0&channel=0
【PaddlePaddle+OpenVINO】PP-HumanSeg部署
OpenVINO™ 工具套件是用于快速开发应用程序和解决方案，以解决各种任务（包括人类视觉模拟、自动语音识别、自然语言处理和推荐系统等）的综合工具套件。该工具套件基于最新一代的人工神经网络，包括卷积神经网络 (CNN)、递归网络和基于注意力的网络，可扩展跨英特尔® 硬件的计算机视觉和非视觉工作负载，从而最大限度地提高性能。它通过从边缘到云部署的高性能、人工智能和深度学习推理来为应用程序加速。

本项目将使用openvino部署paddleseg的PP-Humanseg模型，快速体验paddlepaddle模型使用openvino部署流程。

一、PaddleSeg人像分割
人像分割是图像分割领域的高频应用，PaddleSeg推出在大规模人像数据上训练的人像分割系列模型PP-HumanSeg，包括超轻量级模型PP-HumanSeg-Lite，满足在服务端、移动端、Web端多种使用场景的需求。PaddleSeg提供从训练到部署的全流程应用指南，以及视频流人像分割、背景替换教程。可基于Paddle.js在网页体验人像扣图效果、视频背景替换及弹幕穿透效果。

本次部署实践的效果如下所示。原图：

分割效果：

PaddleSeg人像分割模型使用非常简单，官方已经提供多个预训练模型。可按照以下步骤快速体验。

In [ ]

step 1: git clone paddleseg

%cd ~/
!git clone https://gitee.com/PaddlePaddle/PaddleSeg.git
In [ ]

step 2: install paddleseg

%cd ~/PaddleSeg/
!pip install -e .
!pip install scikit-image
In [ ]

step 3: prepare dataset --如果部署官方模型，可忽略此步骤

%cd ~/PaddleSeg/contrib/PP-HumanSeg/data
!python download_data.py
In [ ]

step 4: download pretrained model

%cd ~/PaddleSeg/contrib/PP-HumanSeg/pretrained_model/
!python download_pretrained_model.py
!wget https://paddleseg.bj.bcebos.com/dygraph/humanseg/train/pphumanseg_lite_generic_192x192.zip
!unzip -oq pphumanseg_lite_generic_192x192.zip
In [ ]

step 5: train --这一步可以跳过，使用官方提供的权重

%cd ~/PaddleSeg/contrib/PP-HumanSeg/
!python train.py
–config configs/fcn_hrnetw18_small_v1_humanseg_192x192_mini_supervisely.yml
–do_eval
–use_vdl
–save_interval 100
–iters 100
–num_workers 4
–save_dir output
In [2]

step 6: export model —导出第四步下载的模型，如果是自己训练的话，更改下方的model_path

%cd ~/PaddleSeg/contrib/PP-HumanSeg/
!python …/…/export.py
–config configs/fcn_hrnetw18_small_v1_humanseg_192x192_mini_supervisely.yml
–model_path pretrained_model/fcn_hrnetw18_small_v1_humanseg_192x192/model.pdparams
–save_dir output/export/fcn_hrnetw18_small_v1_humanseg_192x192
–with_softmax --input_shape 1 3 192 192
/home/aistudio/PaddleSeg/contrib/PP-HumanSeg
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/setuptools/depends.py:2: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module’s documentation for alternative uses
import imp
2022-04-06 18:32:25 [INFO] Loading pretrained model from pretrained_model/fcn_hrnetw18_small_v1_humanseg_192x192/model.pdparams
2022-04-06 18:32:25 [INFO] There are 363/363 variables loaded into FCN.
2022-04-06 18:32:25 [INFO] Loaded trained params of model successfully.
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/utils.py:77: DeprecationWarning: Using or importing the ABCs from ‘collections’ instead of from ‘collections.abc’ is deprecated, and in 3.8 it will stop working
return (isinstance(seq, collections.Sequence) and
2022-04-06 18:32:30 [INFO] Model is saved in output/export/fcn_hrnetw18_small_v1_humanseg_192x192.
In [29]

step 7: predict

%cd ~/PaddleSeg/contrib/PP-HumanSeg/
!python predict.py
–config configs/fcn_hrnetw18_small_v1_humanseg_192x192_mini_supervisely.yml
–model_path pretrained_model/fcn_hrnetw18_small_v1_humanseg_192x192/model.pdparams
–image_path /home/aistudio/test1.jpeg
–save_dir /home/aistudio/result
/home/aistudio/PaddleSeg/contrib/PP-HumanSeg
2022-04-02 10:25:41 [INFO]
---------------Config Information---------------
batch_size: 64
export:
transforms:

target_size:
- 192
- 192
  type: Resize
type: Normalize
iters: 2000
loss:
coef:
1
types:
type: CrossEntropyLoss
lr_scheduler:
end_lr: 0
learning_rate: 0.05
power: 0.9
type: PolynomialDecay
model:
backbone:
align_corners: false
type: HRNet_W18_Small_V1
backbone_indices:
-1
num_classes: 2
pretrained: pretrained_model/fcn_hrnetw18_small_v1_humanseg_192x192/model.pdparams
type: FCN
optimizer:
momentum: 0.9
type: sgd
weight_decay: 0.0005
train_dataset:
dataset_root: data/mini_supervisely
mode: train
num_classes: 2
train_path: data/mini_supervisely/train.txt
transforms:
target_size:
- 192
- 192
  type: Resize
type: RandomHorizontalFlip
brightness_range: 0.4
contrast_range: 0.4
saturation_range: 0.4
type: RandomDistort
type: Normalize
type: Dataset
val_dataset:
dataset_root: data/mini_supervisely
mode: val
num_classes: 2
transforms:
target_size:
- 192
- 192
  type: Resize
type: Normalize
type: Dataset
val_path: data/mini_supervisely/val.txt

W0402 10:25:41.511114 9501 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1
W0402 10:25:41.511162 9501 device_context.cc:465] device: 0, cuDNN Version: 7.6.
2022-04-02 10:25:46 [INFO] Loading pretrained model from pretrained_model/fcn_hrnetw18_small_v1_humanseg_192x192/model.pdparams
2022-04-02 10:25:46 [INFO] There are 363/363 variables loaded into FCN.
2022-04-02 10:25:46 [INFO] Number of predict images = 1
2022-04-02 10:25:46 [INFO] Loading pretrained model from pretrained_model/fcn_hrnetw18_small_v1_humanseg_192x192/model.pdparams
2022-04-02 10:25:46 [INFO] There are 363/363 variables loaded into FCN.
2022-04-02 10:25:46 [INFO] Start to predict…
1/1 [==============================] - 0s 125ms/step
In [30]

step 8: 查看分割结果

import matplotlib.pyplot as plt
import cv2

src_path = “/home/aistudio/test1.jpeg”
dst_path = “/home/aistudio/result/added_prediction/test1.jpeg”

src_image = cv2.imread(src_path)
src_image = cv2.cvtColor(src_image, cv2.COLOR_BGR2RGB)
dst_image = cv2.imread(dst_path)
dst_image = cv2.cvtColor(dst_image, cv2.COLOR_BGR2RGB)

plt.subplot(1, 2, 1)
plt.imshow(src_image)
plt.subplot(1, 2, 2)
plt.imshow(dst_image)
<matplotlib.image.AxesImage at 0x7f5b21a6cd90>

二、openvino部署在模型部署阶段，需要实现数据的预处理。打开~/PaddleSeg/contrib/PP-HumanSeg/output/export/fcn_hrnetw18_small_v1_humanseg_192x192文件夹，查看deploy.yml，其内容如下：

Deploy:
model: model.pdmodel
params: model.pdiparams
transforms:

target_size:
- 192
- 192
  type: Resize
type: Normalize
transforms部分就是部署模型时需要实现的预处理过程。

In [4]

step 1: 安装openvino

!pip install openvino
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: openvino in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (2021.4.2)
Requirement already satisfied: numpy<1.20,>=1.16.6 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from openvino) (1.19.5)
In [ ]

step 2: 预测代码—aistudio 使用openvino 好像有问题，在本地使用

import cv2
import openvino
import argparse
import numpy as np
from openvino.runtime import Core

对应deploy.yml的Normalize

def normalize(im, mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]):
im = im.astype(np.float32, copy=False) / 255.0
im -= mean
im /= std
return im

对应deploy.yml的Resize

def resize(im, target_size=608, interp=cv2.INTER_LINEAR):
if isinstance(target_size, list) or isinstance(target_size, tuple):
w = target_size[0]
h = target_size[1]
else:
w = target_size
h = target_size
im = cv2.resize(im, (w, h), interpolation=interp)
return im

class Predictor:
def init(self, model_path, target_size=(192, 192), mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]):
self.target_size = target_size
self.mean = mean
self.std = std
self.model_path = model_path

def preprocess(self, image):
    # 预处理包含resize和normalize，其参数要和训练时设置的值一致
    image = resize(image, target_size=self.target_size)
    image = normalize(image, mean=self.mean, std=self.std)
    return image
    
def predict(self, image_path):
    # 预处理
    image = cv2.imread(image_path)
    image_h, image_w, _ = image.shape
    rgb_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)  
    inputs = self.preprocess(rgb_image)

    # 预测 
    input_image = np.expand_dims(
        inputs.transpose(2, 0, 1), 0
    )
    ie = Core()
    model = ie.read_model(model=self.model_path)  # openvino支持直接读取paddlepaddle *.pdmodel
    compiled_model = ie.compile_model(model=model, device_name="CPU")
    input_layer_ir = next(iter(compiled_model.inputs))
    output_layer_ir = next(iter(compiled_model.outputs))
    
    mask = compiled_model([input_image])[output_layer_ir][0] # 输出的mask维度为[N, 2, H, W]，是softmax输出的概率
    mask = np.argmax(mask, axis=0) # 获得类别索引
    mask = cv2.resize(mask.astype('uint8'), (image_w, image_h), interpolation=cv2.INTER_NEAREST) # 将mask resize回图片原本的尺寸
    image[mask < 1, :] = 0  # 将非人像部分设置为黑色
    cv2.imshow('1', image)
    cv2.waitKey(0)

if name == “main”:
model_path = “/home/aistudio/test1.jpeg”
image_path = “/home/aistudio/PaddleSeg/contrib/PP-HumanSeg/output/export/model.pdmodel” # 导出的模型路径

predictor = Predictor(model_path, target_size=(192, 192), mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
predictor.predict(image_path)

In [19]

step 3: 查看分割结果

import matplotlib.pyplot as plt
import cv2

src_path = “/home/aistudio/test1.jpeg”
dst_path = “/home/aistudio/result.png”

src_image = cv2.imread(src_path)
src_image = cv2.cvtColor(src_image, cv2.COLOR_BGR2RGB)
dst_image = cv2.imread(dst_path)
dst_image = cv2.cvtColor(dst_image, cv2.COLOR_BGR2RGB)

plt.subplot(1, 2, 1)
plt.imshow(src_image)
plt.subplot(1, 2, 2)
plt.imshow(dst_image)

<matplotlib.image.AxesImage at 0x7f5b2192a090>

三、参考链接 1、轻量级人像分割深度学习模型PP-HumanSeg树莓派部署 2、人像分割PP-HumanSeg模型onnx C++ windows部署

请点击此处查看本环境基本用法.
Please click here for more detailed instructions.