When upgrading from PSReadLine 2.1 to 2.2, Pasting on remote machines gets slow towards end

[wonkifier]

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest released version
• Search the existing issues, especially the pinned issues.

Exception report

N/A

Screenshot

SlowPaste220b5.mov

Environment data

PS Version: 7.3.9
PS HostName: ConsoleHost
PSReadLine Version: 2.2.0-beta5
PSReadLine EditMode: Emacs
OS: Linux <redacted> 3.10.0-1160.99.1.el7.x86_64 #1 SMP Thu Aug 10 10:46:21 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux
BufferWidth: 176
BufferHeight: 62

I first observed this on PSReadLine 2.2.6 (as part of a default Powershell 7.3.9 install), but tested with current version, and then tested back to 2.2.0-beta5.
2.2.0-beta4 pasted normally (everything in one shot, no slowdown for latter portion of paste)

Something changed from beta4 to beta5 that causes this

Steps to reproduce

#- Create a text file/window with the following sample contents to copy from

&{

  Write-Host "Test line"
  Write-Host "Test line"
  Write-Host "Test line"
  Write-Host "Test line"
  Write-Host "Test line"

• From my local Mac running Terminal.app, SSH into the RHEL 8 host in a bash shell
• run pwsh
• Copy the text from the file above and paste into the window
• Observe that the first portion of the paste is instant, then the latter portion appears as if it's being type out by hand
• Press Ctrl-C to get back to a regular prompt

---

Steps I used to change version of PSReadline:

• While logged into remote bash session:

pwsh -noprofile -command "Uninstall-Module PSReadLine"
pwsh -noprofile -command "Install-Module PSReadLine -Force -SkipPublisherCheck -RequiredVersion 2.2.0-beta4 -AllowPrerelease"
pwsh
get-module psreadline # to confirm the installed version

Expected behavior

The entire contents of the past should be "instant", no slowdown at the end.

Actual behavior

Refer to attached video... The first portion of the paste is instant, then the latter portion appears as if it's being typed out by hand

[StevenBucher98]

Looks like your video may have been cut off as its <1 second for me, however I dont seem to have slow pasting on my Mac PSReadLine version 2.3.4, can you see if 2.3.4 also has this issue? Perhaps there is some delay since you are remoting in?

[wonkifier]

Same behavior on 2.3.4.

It can't be just a matter of being a remote connection alone, because it works fine on 2.2.0-beta4 and earlier.

Also, if it were a relatively consistent delay through the entire paste, I could see that as being "remote" caused. But that's not the case. It's "first chunk = full speed" followed by "last several characters one at a time"

But the behavior I'm seeing isn't that.
When I paste the above text, it all goes instantly in one chunk ending with Write-Host "Tes of the last line. Then I see the letter t show up, Then a space, then an l, then an i, then an n, then an e, then a quote.

And if I copy/paste the lines so there are 50 lines instead of 5, it's the same exact behavior... all lines go instantly until after the Tes of the final line, then it finishes one character at a time.
If I extend it to 500 lines, the only difference is what you'd expect when pasting that much text... a couple screenfuls instantly, a pause, some more screenfuls, pause... until that last Tes, after which it completes 1 character at a time.

Every published version from 2.2.0-beta5 has this behavior. No published version before it has it.

Not sure what the deal is with the video... if you right-click to download it and you view it locally does it also get stuck at the first frame?

[wonkifier]

I've tried to test the exact scenario locally (without a remote host) but I can't replicate it exactly on my M1 Mac.

If I use the same docker image with Powershell 7.3 or above, it fails because of .Net incompatibility with Rosetta.
If I run Powershell 7.2, I can't get it accept newer versions of PSReadLine

And if I build a container locally that doesn't require rosetta, it works fine and snappily... except since it's not running on a different platform, I can't rule out the "remote" angle, since I'd need the same platform on both sides.
And I don't have a remote aarch64/arm64 host to try the same image on.

[wonkifier]

I was able to track down a Windows host where I could install Docker and run the same image, and I didn't see the "instant start, slow finish" behavior. So this does seem like something that changed between 2.2.0-beta4 and 2.2.0-beta5 impacted remote session behavior.

I just tested is in iTerm2 on my Mac, so it's not Terminal.app specific. I don't have a handy remote host other than RHEL8 based to test with, so I don't know if it's sensitive to Linux distro.

[mhintzke]

@StevenBucher98

I wrote up an issue in PowerShell repository before realizing (with the help of a Microsoft Engineer) that the root of the issue lied within the PSReadLine module.

Basically we are seeing the exact same think as @wonkifier. If you see my screenshots in the issue linked above, you will see that the websocket messages being sent to and from the pwsh process are significantly different in v2.2.0-beta5 than v2.2.0-beta4. This increase in traffic results in a "lag" to the user which is exacerbated if they are already in a high latency situation (e.g. client is geographically distant from the hosted machine).

I plan to review the changes made there to see if I can spot what could have caused this, but my guess is its by-design for some other requirement and so a "fix" might not be entirely trivial.

[mhintzke]

@daxian-dbw

Hi Dongbo! I know this code was written a while ago, but I am curious if you could take a look at this issue (also see my write-up in this issue) and see if in of the changes you made here line up with the behavior we are seeing. Appreciate any insight into whether or not this is truly a bug or something by-design that will not be fixed.

Thanks

[daxian-dbw]

@mhintzke Thanks for the detailed issue description in PowerShell/PowerShell#24696, especially the screenshots of the before vs. after comparison.

I believe this regression is caused by this line of code in the ReallyRender method, which was added in #3074 to read the current cursor's position.

PSReadLine/PSReadLine/Render.cs

Line 931 in 9e946af

_previousRender.UpdateConsoleInfo(_console);

The PTY on server side should be able to buffer the writes to terminal, but it has to send out all buffered bytes when there is a READ request, such as querying for cursor position. Can you try the changes in daxian-dbw@1071070 (the lagging branch) and see if it mitigates the issue?

[mhintzke]

@daxian-dbw This makes sense and would definitely correlate to the usage of "copy/paste" since the cursor will move 1 space each character that is inserted into the terminal. I can definitely give this a try, I will just need some time to build the module and get your change into our system. Appreciate you taking a look for us! I will update here ASAP.

[mhintzke]

@daxian-dbw I have tested your changes in our environment and I'm afraid the issue still exists. (see results in image of Get-Module showing 2.4.0 beta version of PSReadLine which is your commit)

Here is a GIF of the behavior still. First I copy paste a small string of text and you can see the "lag". Then I remove it and paste in a longer string of text. For some reason, only the last couple of characters appear to be slow. This seems to be constant as I can paste in a very large string and 95% of it will show up instantly, and the last 5% "lag" and come in slowly.

For reference, here is the same test using 2.1.0

Please let me know if there is any more I can do to help you troubleshoot!

[daxian-dbw]

Thanks @mhintzke for the follow-up. Can you share another screenshot of the picture 1 (the captured traffic) with the "data" section to be fully displayed?

One question, when you paste the string, do you use ctrl+v or right clicking the mouse? If it's ctrl+v, I'm trying to understand whether the server-side PSReadLine will receive the ctrl+v key pressing or whether the server-side PTY just sends the letters one by one to PSReadLine as if it's typing really fast?

[mhintzke]

@daxian-dbw Here are 2 screenshots showing the entire set websocket messages from pasting in abcdefhijklmnopq using Shift + Insert (Ctrl + V doesn't work I believe due to it being a unix env). I also confimed that right clicking and clicking on "Paste" in context menu results in the same behavior.

And here is my best attempt at copying the raw data in case you want to use it at all.

42/terminal,["data","\u001b[9;29R"] | 35 | 08:45:18.640 | -- | -- | -- | --
42/terminal,["data","\u001b[?25l\u001b[9;28H\u001b[0m\u001b[92mabcdef\u001b[39;49m\u001b[0m\u001b[9;34H\u001b[?12l\u001b[?25h\u001b[6n"] | 136 | 08:45:18.680 |  
42/terminal,["data","\u001b[9;34R"] | 35 | 08:45:18.681 |  
42/terminal,["data","\u001b[?25l\u001b[9;28H\u001b[0m\u001b[92mabcdefh\u001b[39;49m\u001b[0m\u001b[9;35H\u001b[?12l\u001b[?25h\u001b[6n"] | 137 | 08:45:18.719 |  
42/terminal,["data","\u001b[9;35R"] | 35 | 08:45:18.720 |  
42/terminal,["data","\u001b[?25l\u001b[9;28H\u001b[0m\u001b[92mabcdefhi\u001b[39;49m\u001b[0m\u001b[9;36H\u001b[?12l\u001b[?25h\u001b[6n"] | 138 | 08:45:18.755 |  
42/terminal,["data","\u001b[9;36R"] | 35 | 08:45:18.756 |  
42/terminal,["data","\u001b[?25l\u001b[9;28H\u001b[0m\u001b[92mabcdefhij\u001b[39;49m\u001b[0m\u001b[9;37H\u001b[?12l\u001b[?25h\u001b[6n"] | 139 | 08:45:18.787 |  
42/terminal,["data","\u001b[9;37R"] | 35 | 08:45:18.789 |  
42/terminal,["data","\u001b[?25l\u001b[9;28H\u001b[0m\u001b[92mabcdefhijk\u001b[39;49m\u001b[0m\u001b[9;38H\u001b[?12l\u001b[?25h\u001b[6n"] | 140 | 08:45:18.821 |  
42/terminal,["data","\u001b[9;38R"] | 35 | 08:45:18.822 |  
42/terminal,["data","\u001b[?25l\u001b[9;28H\u001b[0m\u001b[92mabcdefhijkl\u001b[39;49m\u001b[0m\u001b[9;39H\u001b[?12l\u001b[?25h\u001b[6n"] | 141 | 08:45:18.858 |  
42/terminal,["data","\u001b[9;39R"] | 35 | 08:45:18.859 |  
42/terminal,["data","\u001b[?25l\u001b[9;28H\u001b[0m\u001b[92mabcdefhijklm\u001b[39;49m\u001b[0m\u001b[9;40H\u001b[?12l\u001b[?25h\u001b[6n"] | 142 | 08:45:18.892 |  
42/terminal,["data","\u001b[9;40R"] | 35 | 08:45:18.893 |  
42/terminal,["data","\u001b[?25l\u001b[9;28H\u001b[0m\u001b[92mabcdefhijklmn\u001b[39;49m\u001b[0m\u001b[9;41H\u001b[?12l\u001b[?25h\u001b[6n"] | 143 | 08:45:18.924 |  
42/terminal,["data","\u001b[9;41R"] | 35 | 08:45:18.925 |  
42/terminal,["data","\u001b[?25l\u001b[9;28H\u001b[0m\u001b[92mabcdefhijklmno\u001b[39;49m\u001b[0m\u001b[9;42H\u001b[?12l\u001b[?25h\u001b[6n"] | 144 | 08:45:18.955 |  
42/terminal,["data","\u001b[9;42R"] | 35 | 08:45:18.956 |  
42/terminal,["data","\u001b[?25l\u001b[9;28H\u001b[0m\u001b[92mabcdefhijklmnop\u001b[39;49m\u001b[0m\u001b[9;43H\u001b[?12l\u001b[?25h\u001b[6n"] | 145 | 08:45:18.985 |  
42/terminal,["data","\u001b[9;43R"] | 35 | 08:45:18.986 |  
42/terminal,["data","\u001b[?25l\u001b[9;28H\u001b[0m\u001b[92mabcdefhijklmnopq\u001b[39;49m\u001b[0m\u001b[9;44H\u001b[?12l\u001b[?25h"] | 137 | 08:45:19.019