PoseNet2Scratch

Read this in other languages: English, 日本語

PoseNet2Scratch can detect human pose and add special Scratch blocks that can get x and y position of each part of the body.

It uses PoseNet, a machine learning model , for Real-time Human Pose Estimation. PoseNet advantage is that it can detect human poses quickly and accurately with a normal WebCam without using a special device such as Kinect.

ml5.js, a JavaScript library for machine learning, is used to connect it with Scratch 3.

• Sample Project(nekorider)
• Sample Project(glasses)

Try the customized Scratch 3 that has PoseNet2Scratch as an extension.

If you make a project with using PoseNet2Scratch, please share it with hashtag: #posenet2scratch.

Sample Usage

1. Open https://champierre.github.io/scratch3/ on a browser(Chrome is recommended).

2. Click "Add Extension" button(Folder with "+" mark icon).

3. Select "PoseNet2Scratch" extension.

4. You can use PoseNet2Scratch blocks.

5. Select "Video Sensing" extension, if you want to see the image captured by webcam on the stage screen. This is not mandatory.

6. Add "Glasses" sprite.

7. Make a code like below.

8. You will have the glasses shown at the center of your face(x position of the nose) and at the hight of your eyes(y position of the left eye).

With PoseNet2Scratch extension blocks, you can get x and y position of the following body parts:

• nose
• left eye
• right eye
• left ear
• right ear
• left shoulder
• right shoulder
• left elbow
• right elbow
• left wrist
• right wrist
• left hip
• right hip
• left knee
• right knee
• left ankle
• right ankle