流程记录 / 云原生

Running a joint inference example with KubeEdge Sedna

ShemolJun 10, 2024

流程记录云原生

Following the previous post, KubeEdge, EdgeMesh, and Sedna are already deployed. Next, run the joint inference example from the official docs.

Official docs: https://sedna.readthedocs.io/en/latest/examples/joint_inference/helmet_detection_inference/README.html#

As long as EdgeMesh was deployed correctly, this example shouldn't fail if you follow the docs.

Prepare data and models

Download the small model to the edge

shell

mkdir -p /data/little-model
cd /data/little-model
wget <https://kubeedge.obs.cn-north-1.myhuaweicloud.com/examples/helmet-detection-inference/little-model.tar.gz>
tar -zxvf little-model.tar.gz

Download the large model to the cloud

shell

mkdir -p /data/big-model
cd /data/big-model
wget <https://kubeedge.obs.cn-north-1.myhuaweicloud.com/examples/helmet-detection-inference/big-model.tar.gz>
tar -zxvf big-model.tar.gz

Prepare images

Small-model inference worker: kubeedge/sedna-example-joint-inference-helmet-detection-little:v0.3.0

Large-model inference worker: kubeedge/sedna-example-joint-inference-helmet-detection-big:v0.3.0

shell

git clone <https://github.com/kubeedge/sedna.git>
./examples/build_image.sh joint_inference # 后面加joint_inference就只生成联合推理的镜像，不加的话就把包括联邦学习那些都生成了

If it's slow, what I did was add the following to the image build files (joint-inference-helmet-detection-big.Dockerfile and joint-inference-helmet-detection-little.Dockerfile)

plain text

RUN sed -i s@/archive.ubuntu.com/@/mirrors.aliyun.com/@g /etc/apt/sources.list
RUN apt-get clean
RUN pip config set global.index-url <http://mirrors.aliyun.com/pypi/simple>
RUN pip config set install.trusted-host mirrors.aliyun.com
RUN pip install --upgrade pip

That's adding apt and pip mirrors.

Create the joint inference service

(all kubectl operations are on the cloud)

Create the large-model resource on the cloud

shell

kubectl create -f - <<EOF
apiVersion: sedna.io/v1alpha1
kind:  Model
metadata:
  name: helmet-detection-inference-big-model
  namespace: default
spec:
  url: "/data/big-model/yolov3_darknet.pb"
  format: "pb"
EOF

Create the small-model resource for the edge

shell

kubectl create -f - <<EOF
apiVersion: sedna.io/v1alpha1
kind: Model
metadata:
  name: helmet-detection-inference-little-model
  namespace: default
spec:
  url: "/data/little-model/yolov3_resnet18.pb"
  format: "pb"
EOF

On the edge, create a folder; inference image results all go there:

shell

mkdir -p /joint_inference/output

On the cloud, define env vars CLOUD_NODE and EDGE_NODE

shell

CLOUD_NODE="cloud-node-name"
EDGE_NODE="edge-node-name"

On the cloud, create the joint inference service. I swapped the images for domestic mirrors. File contents:

yaml

kind: JointInferenceService
metadata:
  name: helmet-detection-inference-example
  namespace: default
spec:
  edgeWorker:
    model:
      name: "helmet-detection-inference-little-model"
    hardExampleMining:
      name: "IBT"
      parameters:
        - key: "threshold_img"
          value: "0.9"
        - key: "threshold_box"
          value: "0.9"
    template:
      spec:
        nodeName: $EDGE_NODE
        dnsPolicy: ClusterFirstWithHostNet
        containers:
        - image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/kubeedge/sedna-example-joint-inference-helmet-detection-little:v0.3.0
          imagePullPolicy: IfNotPresent
          name:  little-model
          env:  # user defined environments
          - name: input_shape
            value: "416,736"
          - name: "video_url"
            value: "rtsp://localhost/video"
          - name: "all_examples_inference_output"
            value: "/data/output"
          - name: "hard_example_cloud_inference_output"
            value: "/data/hard_example_cloud_inference_output"
          - name: "hard_example_edge_inference_output"
            value: "/data/hard_example_edge_inference_output"
          resources:  # user defined resources
            requests:
              memory: 64M
              cpu: 100m
            limits:
              memory: 2Gi
          volumeMounts:
            - name: outputdir
              mountPath: /data/
        volumes:   # user defined volumes
          - name: outputdir
            hostPath:
              # user must create the directory in host
              path: /joint_inference/output
              type: Directory
  cloudWorker:
    model:
      name: "helmet-detection-inference-big-model"
    template:
      spec:
        nodeName: $CLOUD_NODE
        dnsPolicy: ClusterFirstWithHostNet
        containers:
          - image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/kubeedge/sedna-example-joint-inference-helmet-detection-big:v0.3.0
            name:  big-model
            imagePullPolicy: IfNotPresent
            env:  # user defined environments
              - name: "input_shape"
                value: "544,544"
            resources:  # user defined resources
              requests:
                memory: 2Gi
EOF

Simulate a video stream on the edge for inference

1. Install the open-source video streaming server EasyDarwin.
1. Start the EasyDarwin server.
1. Download the video.
1. Push the stream to a URL the inference service can reach (e.g. rtsp://localhost/video).

(The URL in the docs for EasyDarwin-linux-8.1.0-1901141151.tar.gz is probably gone, but I found it on some site and downloaded it)

shell

cd EasyDarwin-linux-8.1.0-1901141151
./start.sh

mkdir -p /data/video
cd /data/video
wget <https://kubeedge.obs.cn-north-1.myhuaweicloud.com/examples/helmet-detection-inference/video.tar.gz>
tar -zxvf video.tar.gz

ffmpeg -re -i /data/video/video.mp4 -vcodec libx264 -f rtsp rtsp://localhost/video

If it's running normally, the pods are all running, and you can check inference results in the output path defined in the JointInferenceService config (e.g. /joint_inference/output).