Record video/photo using MMAL

[diegobernardes]

Hi, nice project! I think it's the only one for Go that it's actually working. Do you have any snippet to record videos and photos to files using MMAL? The original documentation for MMAL is really bad and I'm having a bad time trying to figure it out.

[djthorpe]

Hi Diego, it's not quite working yet I'm afraid - I still have some work to do for recoding of videos and photo files, but I do plan to add that sometime.

The example code I'm working on so far is here:

https://github.com/djthorpe/gopi-hw/tree/master/cmd

I am not sure these are all working yet (it's been a while since I looked at this code).
Please feel free to ask questions on implementing something yourself, or I will get around to this sometime in the next few weeks.

[diegobernardes]

Hi David,

It's ok, I'll try to figure out how I can do this, maybe I can contribute to the project. I actually did something with OpenCV that uses V4L2 and behind the scenes it uses MMAL, but at the end I can't take all the advantage features available. I'll try to do something with gopi and maybe even contribute with something. The problem is the MMAL documentation, it's really bad and complex.

[djthorpe]

Hey Diego, let’s leave this issue open then and we can use it to ask and answer questions. I’d be really happy to assist you with getting this working. I know of someone else who asked the same question recently and would be interested. Really happy to accept pull requests and help with testing too.

I’ll also take a look at the code and see what insights I have on getting this working.

[djthorpe]

Hi Diago, I spent a couple of hours and I now have the skeleton code for capturing images from the camera. I can get a very small JPEG out from the camera, encoded by the Image encoder coimponent. The code is here in the v1 branch of the gopi-hw repository:

https://github.com/djthorpe/gopi-hw/tree/v1/cmd/mmal_capture

In order to check the code out and build it, you can do the following:

git clone [email protected]:djthorpe/gopi-hw.git
cd gopi-hw
git checkout v1
PKG_CONFIG_PATH="/opt/vc/lib/pkgconfig" go run -tags "rpi"  ./cmd/mmal_capture -debug

Basically the code is a mash up of the other two commands, mmal_camera_preview and mmal_encode_image. The main loop which reads the data from the encoder component is something like this, with all the error handing and setup removed:

for {
	buffer, err := this.encoder.GetEmptyBufferOnPort(port_out, true)
        port_out.Send(buffer)
        buffer, err := this.encoder.GetFullBufferOnPort(port_out, true)  
        writer.Write(buffer.Data())
        eos = buffer.Flags()&hw.MMAL_BUFFER_FLAG_EOS != 0
	port_out.Release(buffer)
        if(eos) { break }
}

It does create a tiny JPG on disk, but the loop doesn't end. In the log files I see the following:

[DEBUG] GetEmptyBufferOnPort
[DEBUG] GetFullBufferOnPort
[DEBUG] OUTPUT EVENT: vc.ril.image_encode:out:0(JPEG): buffer=<MMAL_Buffer>{ alloc_size=81920 length=1382 flags=MMAL_BUFFER_FLAG_FRAME_END } => <MMAL_Queue>{ length=0 }
[DEBUG] POOL EVENT: vc.ril.image_encode:out:0(JPEG): buffer=<MMAL_Buffer>{ alloc_size=81920 length=0 }
false <MMAL_Buffer>{ alloc_size=81920 length=0 }
[DEBUG] GetEmptyBufferOnPort
[DEBUG] GetFullBufferOnPort
[DEBUG] OUTPUT EVENT: vc.ril.image_encode:out:0(JPEG): buffer=<MMAL_Buffer>{ alloc_size=81920 length=0 flags=MMAL_BUFFER_FLAG_EOS } => <MMAL_Queue>{ length=0 }

You can see that the MMAL_BUFFER_FLAG_EOS (end of stream) flag is definitely coming out. I am sure if I had another couple of hours it could really work well. In addition, there are no parameters set on the encoder, camera or renderer components, so you have absolutely no control over the image except it encodes a very small JPEG.

Perhaps this code is a good starting point for you? I would love you to take a look at this code and turn it into something actually useful for still image capture! Getting still images working well and with configuration parameters seems to be achievable at least within a few hours of work I think. Please let me know if you are also able to get this functioning, and if you do enhance it send me a pull request or questions and I'd love to incorporate into the code.

Here are some sources of information you might want to tap into. Particularly, there's a lot of really high quality documentation for picamera which is a Python module:

• https://github.com/waveform80/picamera
• https://github.com/raspberrypi/userland/tree/2448644657e5fbfd82299416d218396ee1115ece/host_applications/linux/apps/raspicam
• https://www.raspberrypi.org/documentation/usage/camera/raspicam/raspistill.md
• https://picamera.readthedocs.io/en/release-1.13/
• https://picamera.readthedocs.io/en/release-1.13/api_mmalobj.html

Any questions or comments, just shout!

[diegobernardes]

Really nice! I'll try to do something for sure, thanks for this example. Just need to organize my self with all the side projects running at the same time 😅

[djthorpe]

Yeah, I also have a million side projects going. So I have re-organized the code a bit. Eventually the "Camera" would end up in the gopi-media repository which is where I would like media (audio, video, encoding, etc) to live but for the moment I've made a camera interface in the gopi-hw repository:

type Camera interface {
	gopi.Driver

	// Get Camera Properties
	CameraId() uint32
	CameraModel() string
	CameraFrameSize() gopi.Size

	// Read and write the camera configuration
	CameraConfig() (CameraConfig, error)
	SetCameraConfig(CameraConfig) error

	// Preview, image and video capture
	Preview() error
	ImageCapture() error
}

The command line app in the cmd/mmal_capture is still the one which takes config parameters. The parameters are pretty basic at the moment:

Usage of mmal_capture:
  -camera.id uint
    	Camera selection
  -preview
    	Display capture preview
  -rotate int
    	Rotation angle (0,90,180 or 270)

It doesn't yet write the image out to a file, nor does it always work, sometimes it hangs which means I'm not correctly shutting down the MMAL system. I likely won't be able to spend much more time on this for a little while...