1.14.1 Paper #26 Server Hang

[ichbinashh]

https://pastebin.com/u9awiRez

[RoboMWM]

I should find a bot to unravel small pastebin links.

15:48:01 ERROR]: --- DO NOT REPORT THIS TO PAPER - THIS IS NOT A BUG OR A CRASH  - git-Paper-26 (MC: 1.14.1) ---
[15:48:01 ERROR]: The server has not responded for 25 seconds! Creating thread dump
[15:48:01 ERROR]: ------------------------------
[15:48:01 ERROR]: Server thread dump (Look for plugins here before reporting to Paper!):
[15:48:01 ERROR]: ------------------------------
[15:48:01 ERROR]: Current Thread: Server thread
[15:48:01 ERROR]:       PID: 20 | Suspended: false | Native: false | State: TIMED_WAITING
[15:48:01 ERROR]:       Stack:
[15:48:01 ERROR]:               sun.misc.Unsafe.park(Native Method)
[15:48:01 ERROR]:               java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
[15:48:01 ERROR]:               net.minecraft.server.v1_14_R1.IAsyncTaskHandler.awaitTasks(IAsyncTaskHandler.java:116)
[15:48:01 ERROR]:               net.minecraft.server.v1_14_R1.ChunkProviderServer.getChunkAt(ChunkProviderServer.java:96)
[15:48:01 ERROR]:               net.minecraft.server.v1_14_R1.IChunkProvider.getChunkAt(SourceFile:16)
[15:48:01 ERROR]:               org.bukkit.craftbukkit.v1_14_R1.CraftWorld.getChunkAt(CraftWorld.java:375)
[15:48:01 ERROR]:               me.ryanhamshire.PopulationDensity.PopulationDensity.scanRegion(PopulationDensity.java:1286)
[15:48:01 ERROR]:               me.ryanhamshire.PopulationDensity.ScanOpenRegionTask.run(ScanOpenRegionTask.java:27)
[15:48:01 ERROR]:               org.bukkit.craftbukkit.v1_14_R1.scheduler.CraftTask.run(CraftTask.java:84)
[15:48:01 ERROR]:               org.bukkit.craftbukkit.v1_14_R1.scheduler.CraftScheduler.mainThreadHeartbeat(CraftScheduler.java:452)
[15:48:01 ERROR]:               net.minecraft.server.v1_14_R1.MinecraftServer.b(MinecraftServer.java:1111)
[15:48:01 ERROR]:               net.minecraft.server.v1_14_R1.DedicatedServer.b(DedicatedServer.java:418)
[15:48:01 ERROR]:               net.minecraft.server.v1_14_R1.MinecraftServer.a(MinecraftServer.java:1057)
[15:48:01 ERROR]:               net.minecraft.server.v1_14_R1.MinecraftServer.run(MinecraftServer.java:902)
[15:48:01 ERROR]:               java.lang.Thread.run(Unknown Source)
[15:48:01 ERROR]: ------------------------------
[15:48:01 ERROR]: --- DO NOT REPORT THIS TO PAPER - THIS IS NOT A BUG OR A CRASH ---
[15:48:01 ERROR]: ------------------------------

[ichbinashh]

As with the other issue, the latest version of Paper seems to have fixed this

[RoboMWM]

This is due to chunk loading, if not chunk generation which is why you're probably not having an issue now.

[smellyonionman]

Having the same on Paper 68, which is 1.14.2. I have to say it seems like plugin authors are actually farther ahead of Mojang, with 1.14. Really surprising lately.

[RoboMWM]

This occurred during a region scan, and likely was taking a while since it was generating chunks. This could be refactored to be split up over a few ticks, or "asynchronously" with Paper's async chunk load API. Both will take a fair bit of time to do, though I guess PaperLib can help the latter refactor with compatibility a lot easier.

[RoboMWM]

The reason why scanRegion takes longer is there currently is no async chunk loading at all in paper or spigot, and 1.14 chunk loading/generation is a lot heavier than previous versions from what I hear.

So there's no use (right now) to use Paper's async chunk loading API as it will not load chunks asynchronously for the moment.

I'll look into breaking up the task to take chunks in intervals - which will likely be a significant refactor unfortunately.

The actual scan is done asynchronously, but it needs snapshots of the chunks in order to perform the scan. The current design is to acquire all chunk snapshots within a region, which first requires each chunk to be loaded - and this has to be done synchronously. This is currently all done in a single task (and as such, within a single tick), hence why it'll take work to break this up into intervals.

[RoboMWM]

If anyone's interested in PRing, here's the relevant task: https://github.com/TechFortress/PopulationDensity/blob/5b5456d34a48d50ba18012a1d7b68026b8a9982e/src/me/ryanhamshire/PopulationDensity/PopulationDensity.java#L1234-L1280

[RoboMWM]

@ichbinashh What was the output of /version PopulationDensity you were using when you reported this issue?

[Spoonerj1]

any luck on this issue? I had to remove the plugin for a bit because it wouldnt play nice with 1.14.3. Would love to bring it back

[RoboMWM]

Pregenerating the world would help a lot, else the scanner will generate chunks, which takes much longer to perform than loading.

[yashendra2797]

Any progress? Population Density drags down my TPS from 20 to 16.

[RoboMWM]

@yashendra2797

Pregenerating the world would help a lot, else the scanner will generate chunks, which takes much longer to perform than loading.

Also, you're looking at average TPS from a server start, which isn't a good metric to be taking statistics for performance on.

[yashendra2797]

@yashendra2797

Pregenerating the world would help a lot, else the scanner will generate chunks, which takes much longer to perform than loading.

Also, you're looking at average TPS from a server start, which isn't a good metric to be taking statistics for performance on.

I did pregen the world using Worldborder.

How long should, in an ideal scenario PopulationDensity affect TPS before it goes back to normal?

[RoboMWM]

Only the first phase of the scanner, where it obtains the ChunkSnapshots. The rest of the scanning is done asynchronously. Iirc there's an option to turn off the scan at startup? But PD only affects maybe half a minute if it doesn't need to close a region.

[yashendra2797]

Alright, I'm gonna turn on Population Density next time when we have players playing, and leave it running for 2 hours to judge average TPS.

[RoboMWM]

Use /timings paste after the 2 hours to get results as to what is actually taking up CPU time.