Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PS3 Microphone array support #51

Open
wants to merge 40 commits into
base: master
Choose a base branch
from

Conversation

marcel303
Copy link

@marcel303 marcel303 commented Jul 24, 2018

Hi! I added a USB audio class driver implementation for the PS3 eye camera's microphone array. This allows one to record both video (using the existing video streaming code from PS3EYEDriver) and audio. It adds a new class, PS3EYEMic, which can be optionally used alongside PS3EYECam.

See the code in action here: https://youtu.be/o1iWJ17HWI8
The fifth, white, channel, is the average of the four microphone capsules. Just averaging them adds a nice noise filtering effect. :-)

Audio capture is performed for all four channels of the PS3 Eye, at 16kHz (the only sample rate it seems to support, despite marketing material advertising 48kHz.)

I also fixed a few issue in the existing PS3EYECam code, as it was stopping me from getting this to work (debayer fix, request free fix). The pull request includes these fixes.

Init:

	PS3EYEMic mic;
	MyAudioCallback audioCallback;
	
	if (eye != nullptr)
	{
		mic.init(eye->getDevice(), &audioCallback);
	}

MyAudioCallback:

	struct MyAudioCallback : AudioCallback
	{
		virtual void handleAudioData(const AudioFrame * frames, const int numFrames) override
		{
			// AudioFrame contains four int16_t's, one for each mic in the array.
			// Do something with this data here..
		}
	};

Regards,
Marcel Smit

bakercp and others added 30 commits December 13, 2017 00:17
…where library user needs to have libusb include folder in path. With this change only the PS3EYEDriver need to know about libusb.
…ure library user doesn't get a conflicting define for 'debug', which is pretty generic.
…river for the PS3 eye camera, preventing us from using the device otherwise.
- Shutdown on init failure.
…t to stop calling submit_transfer.

- Renamed cancelTransfers to endTransfers.
- Added grayscale test to see if crash after mic test in debayer code persists..
- Added some comments.
- Improved shutdown in case of error.
- Use libusb error/success codes instead of comparing against 0.
- Code cleanups and readability improvenents.
- Made a few constant 'static const int' versus a define.
…transfers has reached zero.

- Bumped the packet size to 256 bytes, to avoid overflow conditions due to the device sending more data than we can store.
- Added logging for the case where a transfer has ended while status is not COMPLETED.
- Drawing of the captured audio waveform is now faster.
- Combined audio and video test. It works!
… it on screen. The summed channel effective does some pretty good noise reduction!
…during shutdown. It's illegal to free a transfer from within the transfer callback. Doing so can result in undefined behavior.

From the docs of libusb_free_transfer,
	"It is not legal to free an active transfer (one which has been submitted and has not yet completed)."

I've moved to free calls until after all transfers have finished.
…rying to debayer one row too many. It should run until height-2, as a single source row expands to two rows in the destination buffer. At the end the last row is already copied from the row at height-2 to height-1.

Verified it works by visually inspecting the last row using a test app.
…' to stop the microphone USB transfers. When encountering an error submitting the new transfer, try a few times until it works. Macos sometimes returns a 'iso too old' result when trying to submit a transfer, which just means we need to transfer again using a more recent frame.
- Show little green/red circles to indicate PS3 eye and mic are up and running.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants