Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ESP-Lyrat-mini-V1.2 recording microphone silent in example raw_http and voice_recognition/vad (AUD-5627) #1252

Open
3 tasks done
HyouinSchoolAcc opened this issue Aug 20, 2024 · 11 comments
Open
3 tasks done

ESP-Lyrat-mini-V1.2 recording microphone silent in example raw_http and voice_recognition/vad (AUD-5627) #1252

HyouinSchoolAcc opened this issue Aug 20, 2024 · 11 comments

Comments

@HyouinSchoolAcc
Copy link

HyouinSchoolAcc commented Aug 20, 2024 •
edited
Loading

Environment

  • Audio development kit: ESP32-LyraT-Mini-1.2
  • [Required] Module or chip used: ESP32-WROVER-E
  • [Required] IDF version v5.2.2
  • [Required] ADF version v2.6-159-gdac74d81
  • Build system: [Make|CMake|idf.py]
  • [Required] Running log:
  • Compiler version (run xtensa-esp32-elf-gcc --version in your project folder to find it):
    // 1.22.0-80-g6c4433a
  • Operating system: [Windows|Linux|macOS]
  • (Windows only) Environment type: [MSYS2 mingw32|ESP Command Prompt|Plain Command Prompt|PowerShell]
  • Using an IDE?: [No|Yes (please give details)]
  • Power supply: [USB|external 5V|external 3.3V|Battery]

Problem Description

When using two examples that utilizes the microphone, I've noticed that all the pipelines are working but the microphone itself is not providing any sound. I bought a second lyrat mini 1.2 to confirm this isnt a unique hardware issue. It seems that the i2s interface is working fine given that the speaker can be played.

Expected Behavior

When the board is programmed the microphone is supposed to pick up sound

Actual Behavior

All sound recorded/inputted are null and the sound audio is completely silent

Steps to Reproduce

  1. get an esp32-lyrat-mini-v1.2
  2. upload raw_http example or the voice_recognition/vad example
  3. in the raw_http example the recorded voice will be blank. In the vad example there will not be any updates after "vad setup and running"

// If possible, attach a picture of your setup/wiring here.

Code to Reproduce This Issue

All code are exactly as written as the example given in examples raw_http and vad.

Debug Logs

I (10) boot: ESP-IDF v5.2.2-dirty 2nd stage bootloader
I (10) boot: compile time Aug 16 2024 �I (10) boot: ESP-IDF v5.2.2-dirty 2nd stage bootloader
I (10) boot: compile time Aug 16 2024 11:49:03
I (10) boot: Multicore bootloader
I (14) boot: chip revision: v3.1
I (18) boot.esp32: SPI Speed : 80MHz
I (23) boot.esp32: SPI Mode : DIO
I (27) boot.esp32: SPI Flash Size : 8MB
I (32) boot: Enabling RNG early entropy source...
I (37) boot: Partition Table:
I (41) boot: ## Label Usage Type ST Offset Length
I (48) boot: 0 nvs WiFi data 01 02 00009000 00006000
I (55) boot: 1 phy_init RF data 01 01 0000f000 00001000
I (63) boot: 2 factory factory app 00 00 00010000 00200000
I (70) boot: End of partition table
I (74) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=2bff0h (180208) map
I (109) esp_image: segment 1: paddr=0003c018 vaddr=3ffb0000 size=03f98h ( 16280) load
I (112) esp_image: segment 2: paddr=0003ffb8 vaddr=40080000 size=00060h ( 96) load
I (115) esp_image: segment 3: paddr=00040020 vaddr=400d0020 size=a7314h (684820) map
I (224) esp_image: segment 4: paddr=000e733c vaddr=40080060 size=1a0cch (106700) load
I (258) boot: Loaded app from partition at offset 0x10000
I (258) boot: Disabling RNG early entropy source...
I (269) cpu_start: Multicore app
I (269) quad_psram: This chip is ESP32-D0WD
I (270) esp_psram: Found 8MB PSRAM device
I (271) esp_psram: Speed: 40MHz
I (275) esp_psram: PSRAM initialized, cache is in low/high (2-core) mode.
W (283) esp_psram: Virtual address not enough for PSRAM, map as much as we can. 4MB is mapped
I (1172) esp_psram: SPI SRAM memory test OK
I (1179) cpu_start: Pro cpu start user code
I (1179) cpu_start: cpu freq: 160000000 Hz
I (1180) cpu_start: Application information:
I (1183) cpu_start: Project name: record_raw_http
I (1188) cpu_start: App version: 1
I (1193) cpu_start: Compile time: Aug 19 2024 21:34:02
I (1199) cpu_start: ELF file SHA256: c6650c452...
I (1205) cpu_start: ESP-IDF: v5.2.2-dirty
I (1210) cpu_start: Min chip rev: v0.0
I (1215) cpu_start: Max chip rev: v3.99
I (1220) cpu_start: Chip rev: v3.1
I (1225) heap_init: Initializing. RAM available for dynamic allocation:
I (1232) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (1238) heap_init: At 3FFB8A00 len 00027600 (157 KiB): DRAM
I (1244) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (1251) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (1257) heap_init: At 4009A12C len 00005ED4 (23 KiB): IRAM
I (1264) esp_psram: Adding pool of 4096K of PSRAM memory to heap allocator
I (1272) spi_flash: detected chip: gd
I (1275) spi_flash: flash io: dio
W (1280) i2c: This driver is an old driver, please migrate your application code to adapt driver/i2c_master.h
W (1290) ADC: legacy driver is deprecated, please migrate to esp_adc/adc_oneshot.h
I (1299) main_task: Started on CPU0
I (1309) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocations
I (1309) main_task: Calling app_main()
I (1359) REC_RAW_HTTP: [ 1 ] Initialize Button Peripheral & Connect to wifi network
W (1459) PERIPH_WIFI: WiFi Event cb, Unhandle event_base:WIFI_EVENT, event_id:41
W (2679) PERIPH_WIFI: WiFi Event cb, Unhandle event_base:WIFI_EVENT, event_id:41
W (2829) PERIPH_WIFI: WiFi Event cb, Unhandle event_base:WIFI_EVENT, event_id:4
I (4329) REC_RAW_HTTP: [ 2 ] Start codec chip
W (4349) I2C_BUS: I2C bus has been already created, [port:0]
I (4369) REC_RAW_HTTP: [3.0] Create audio pipeline for recording
I (4369) REC_RAW_HTTP: [3.1] Create http stream to post data to server
I (4369) REC_RAW_HTTP: [3.2] Create i2s stream to read audio data from codec chip
I (4379) REC_RAW_HTTP: [3.3] Register all elements to audio pipeline
I (4379) REC_RAW_HTTP: [3.4] Link it together [codec_chip]-->i2s_stream->http_stream-->[http_server]
I (4399) REC_RAW_HTTP: Volume set successfully to 100.
I (4399) REC_RAW_HTTP: Current volume is: 100
I (4399) REC_RAW_HTTP: [ 4 ] Press [Rec] button to record, Press [Mode] to exit
W (17339) PERIPH_WIFI: WiFi Event cb, Unhandle event_base:WIFI_EVENT, event_id:41
E (53339) REC_RAW_HTTP: [ * ] [Rec] input key event, resuming pipeline ...
W (53339) AUDIO_PIPELINE: Without stop, st:1
W (53339) AUDIO_PIPELINE: Without wait stop, st:1
W (53339) AUDIO_ELEMENT: [i2s] Element has not create when AUDIO_ELEMENT_TERMINATE
W (53349) AUDIO_ELEMENT: [http] Element has not create when AUDIO_ELEMENT_TERMINATE
W (53359) AUDIO_THREAD: Make sure selected the CONFIG_SPIRAM_BOOT_INIT and CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY by make menuconfig
Total bytes written: 61440
Total bytes written: 64800
I (55429) REC_RAW_HTTP: [ + ] HTTP client HTTP_STREAM_POST_REQUEST, write end chunked marker
I (55439) REC_RAW_HTTP: [ + ] HTTP client HTTP_STREAM_FINISH_REQUEST
W (55449) HTTP_CLIENT: esp_transport_read returned:-1 and errno:128
I (55449) REC_RAW_HTTP: Got HTTP Response = File 20240820T032231Z_16000_16_1.wav was written, size 64800

Other Items If Possible

  • sdkconfig file (Attach the sdkconfig file from your project folder)
  • elf file in the build folder (Note this may contain all the code details and symbols of your project.)
  • coredump (This provides stacks of tasks.)
@github-actions github-actions bot changed the title ESP-Lyrat-mini-V1.2 recording microphone silent in example raw_http and voice_recognition/vad ESP-Lyrat-mini-V1.2 recording microphone silent in example raw_http and voice_recognition/vad (AUD-5627) Aug 20, 2024
@HyouinSchoolAcc
Copy link
Author

sdkconfig.json
record_raw_http.zip
no crashes - coredump not included

@jason-mao
Copy link
Collaborator

@HyouinSchoolAcc Would please test example from latest master branch?

@HyouinSchoolAcc
Copy link
Author

I have tested the latest master branch esp-adf.
My steps:

  1. git pull from adf
  2. delete current workspace copy
  3. import project, copy into workspace
  4. cd into workspace in terminal, idf.py menuconfig -> modify the server address, ssid and password, and config board to lyrat-mini 1.1 board.
  5. flash onto target esp32

The audio returned from the webserver is still silent

@HyouinSchoolAcc
Copy link
Author

HyouinSchoolAcc commented Aug 21, 2024 •
edited
Loading

I'm looking at the diagram and it appears that the I2S peripheral is connected to the ES7243 ADC, and I think it might be helpful to probe the ADC to make sure its responding correctly. I added logI''s to i2s_stream.c under i2s_read but it appears it is not being registered as my logs never show up during recording.

@HyouinSchoolAcc
Copy link
Author

Upon further investigation, it appears that this board, the v1.2, needs its ES7243 initialized with I2C register writes in order to receive audio. I read some documentations in ES7243 and is now able to turn on the ADC chip. I still don't know how it interfaces with the codec chip though.

@Bhavesh-Moradiya
Copy link

I am working on same board and having issue with i2s, I want raw data but because of 2 different IC used in no luck

@Bhavesh-Moradiya
Copy link

I checked BSP packages and found ES8311 I2C address is 0x30 and es7243 0x26 but according to schematic it should be 18 and 20 respectively, if any idea about this please let me know what I am missing

@HyouinSchoolAcc
Copy link
Author

HyouinSchoolAcc commented Aug 26, 2024 via email

@fakefred
Copy link

fakefred commented Oct 3, 2024

I have the same problem, though I've only tried raw_http.

I opened the recorded audio in Ardour (you can also use Audacity), amplified it, then inspected the signal on the spectrogram. It's just white noise all over 0 to 8 kHz.

Screenshot_20241003_155905

I also took a sample of me knocking on the mic with my finger. The peaks do show up, but the gain is super low. I tried setting the volume with audio_hal_set_volume(board_handle->audio_hal, 100), but it didn't help at all.

Screenshot_20241003_160016

I checked BSP packages and found ES8311 I2C address is 0x30 and es7243 0x26 but according to schematic it should be 18 and 20 respectively, if any idea about this please let me know what I am missing

@Bhavesh-Moradiya, I think 0x26 is correct. According to the ES7243 datasheet, the 7-bit address is 0b00100xx where xx is ~AD1 ~AD0. Per the LyraT-mini schematic, the pins AD1 and AD0 are both grounded, so it would be 0b0010011. Add the 0 to the right and we get 0x26.

I read some documentations in ES7243 and is now able to turn on the ADC chip.

@HyouinSchoolAcc, could you tell us more?

@Bhavesh-Moradiya
Copy link

@fakefred

yeah, u are right, I miscalculated the i2c address, I did complete the task recording and playing audio on ESP32 LyraT Mini 1.2

I achieved this using Pipeline and without it, its working good in loop also with TCP audio streaming OUT and IN with PCM Raw byte

@Bhavesh-Moradiya
Copy link

@HyouinSchoolAcc

use below code to initialize both codec IC

    audio_board_handle_t board_handle = audio_board_init();
    audio_hal_ctrl_codec(board_handle->audio_hal, AUDIO_HAL_CODEC_MODE_BOTH, AUDIO_HAL_CTRL_START);
    audio_hal_ctrl_codec(board_handle->adc_hal, AUDIO_HAL_CODEC_MODE_ENCODE, AUDIO_HAL_CTRL_START);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@fakefred @jason-mao @Bhavesh-Moradiya @HyouinSchoolAcc