How to handle simultaneous USB data and serial? Stability issues when using both?

[SeongGino]

Hi! I'm working on a project with the RP2040 in the Arduino IDE (which is more like a fork of an existing one), which when connected with a camera and buttons can make it into a customizable lightgun with force feedback and other bits.
Not trying to be self-promoting, but just giving context - buttons are connected directly to pins, and the camera is a DF Robot IR positioning cam read through I2C, and through some magic processing the IR positioning gets translated to the position of an absolute positioning mouse (a semi-custom library, modified from Arduino absolute positioning mouse library) that the board is acting as (and buttons get sent as mouse/keyboard clicks).

Anyways, getting those working on their own isn't the issue or anything. What I'm running into is, I've come to the point that I want to have the RP2040 read serial commands that are sent to the microcontroller through an app on the PC, in this instance MAMEHOOKER - these vague serial commands (looks something like F2x1x255, which when decoded means to turn on a red LED with a strength of 255) get decoded into a queue and is processed as force feedback commands (manipulating a solenoid or rumble motor if attached) to be synced with game events.

I can, indeed, read serial just fine - however, only when Serial communications are ongoing while the mouse state is being updated, there's about a 20% chance that the board will crash; as in, mouse buttons are still depressed if they were being held already, cursor doesn't respond to movement, and anything else on the board that normally would respond don't anymore. Doesn't react or change state to new serial commands or anything--it's not a crash, it doesn't reset on its own, and I have to physically unplug it to forcibly stop it (and stop my solenoid from heating up if it was engaged).

This doesn't really seem to have anything to do with how I'm processing my Serial buffer--without that in the code, I can replicate this by simply adding a Serial.println("text"); in the main loop, and either waggle the camera or press a wired-up button, and either or will make the board stall similarly. In my mind, this makes no sense, since in this test scenario there's no locking while statements or anything.

So, uh, rambling aside, I've been working at this stability problem alone for the past week and change, and I'm getting exhausted. Is there something I'm missing here? Anything that would normally cause these or similar stability problems?

Unfortunately I can't really provide a minified sketch to demonstrate/test this more easily, as current circumstances means I don't have a spare board and/or camera... I'm really just looking for general advice, if anything, or if this is a problem somehow overlooked in this Arduino core.

[SeongGino]

Update: I looked a little further, and it seems like it's an issue related to the TinyUSB stack I use - see adafruit/Adafruit_TinyUSB_Arduino#238 and adafruit/Adafruit_TinyUSB_Arduino#293, both exhibiting identical, if not the exact same symptoms: the board locking up when data is being sent/received bidirectionally at about the same time.