Add 2 parameters to Start-ThreadJob: timeout seconds, priority #26
Comments
How does a timeout solve this? All you end up with is a section the task run and an indeterminate end state.
If the thread is hung and unresponsive, you won't be able to cancel it with a cooperative cancellation mechanism (eg cancellation is an unwind performed in the thread). Preemptive termination leaves the process in an indeterminate state.
PowerShell is not a RTOS
Is the expectation that cancelling a local thread will also propagate to cancel remote threads, processes or any child local processes? What mechanism should be used to terminate a remote process?
How does a timeouts solve memory leaks? If a thread is leaking memory and you stop it, it has still leaked memory, the memory won't be recovered. If the memory is recovered by the thread termination then it is not actually a leak, it is just memory usage.
100% CPU is not an issue if it is performing the required jobs. Cancelling a thread because you have high CPU just means you aren't performing the jobs you expected it to perform. |
|
@kasini3000 Are you able to implement timed thread cancellation in your own code? For example when you create a thread also create a CancellationToken and call CancellationTokenSource.CancelAfter to set your timeout. Then each thread has a cancellation token, use a static ThreadLocal<> to associate the thread token with each thread when they start. Where ever you have a loop, then also check the state of the CancellationToken associated with the current thread and if cancelled throw an exception. Likewise anywhere that you Thread.Sleep(timeout), do cancellationToken.WaitHandle.WaitOne(timeout) and again throw an exception if the cancellation has occurred. This could be implemented in a small C# class that you use in your PowerShell project. |
|
Likewise, in the same module you can do To adjust the priority. |
Suggest use existing ThreadPriority enum |
|
This is a pattern you can use now and also on original non-core PowerShell.exe |
|
timeoutseconds can make the ps1 script in the thread more robust. I hope ANYONE can provide PR as soon as possible to realize this function |
|
@rhubarb-geek-nz Could you show a small example how to use |
|
The more I look at PowerShell threads the more I think it is better to forget about them actually being threads. I may be completely wrong about this but PowerShell threads are completely isolated from other threads, this is unlike a traditional threading system where threads share memory. Here it appears that threads use their own runspace and are isolated from others. From my tests, even $Global are not global, they seem effectively visible only with that thread/runspace. |
|
That said, if they are threads running in the same process you might be able to pass a cancellation token created in the main thread or any other thread onto another thread via argument list into the parameters of a script block. This would not work for jobs in other process or remote jobs. |
|
@stevenebutler |
news：asp.net8 pr3 add timeout for per requesthttps://devblogs.microsoft.com/dotnet/asp-net-core-updates-in-dotnet-8-preview-3/ You can set request timeouts for individual endpoints, controllers, or dynamically per request. To apply request timeouts, first add the request timeout services: builder.Services.AddRequestTimeouts(); |
I don't really know much about this side of PowerShell so I don't think I can help here. If I read the issue correctly (and I may not) you are looking for something to cancel jobs that are running after a timeout. If that's the case I don't think cancellation tokens are going to help you because they rely on the code being cancelled being co-operative in that it must periodically check for cancellation. If you're stuck inside third party code that does not use/check a cancellation token then it will not take any notice of the cancellation of the token and will continue to process until it finishes. The fix I applied was to make the code base in IWR (which we control) more co-operative by checking the cancellation token in places where it was not. |
sounds good.
Can the job be placed in a subfunction that enforces the use of a cancel token? |
|
The only way I know of to deal with an uncancellable job is to do one of the following
If you're trying to cancel arbitrary code, the safest way to do it is to run it in a separate process and terminate that once the timeout is reached. You should be able to do this already using start-job and related cmdlets. |
Even if you don't want to help PR, don't say that. The thread must be able to be terminated. this issue only want adding a timeout-- If a thread may never be terminated, who dares to use it? golang : “context.WithTimeout” this issue force me to use Python or Golang |
|
Iwr isn't terminating a thread it is cooperatively checking the cancellation token and throwing an exception of it is cancelled. This is the safe way to do cancellation in c#. If you control the code you're running in a thread you can pass it a cancellation token and check it in your code whenever doing a long running operation. If you don't control the code you need it to be checking the cancellation token or it won't cancel cooperatively. The c# APIs all check cancellation token when they're doing io if you use the right overrides (those that accept a token). You can cancel a thread but tasks run on the thread pool which I would assume makes it hard to know which thread to cancel and they aren't designed to work this way. Thread cancellation is designed to work with known threads but I still think using it is not recommended. Note I am not speaking for PowerShell team as I have only just started contributing to it as a community member. I could be wrong and may just not understand what you are trying to do. Unfortunately this is not something I can help with. |
With C#, PowerShell and arbitrary tasks these conditions are not met. |
|
1 powershell has the function of terminating （runspace）threads. So don't tell me that it's not possible or not recommended. Although I don't understand the details, I think ending a thread is easy. Think: Some of the vegetables you eat every day have rotten leaves, and the good parts of the vegetables leaves are also discarded by you. I suggest you endure rotten leaves and not discard them.in other words: Nothing is impossible, only unexpected. |
https://learn.microsoft.com/en-us/dotnet/standard/threading/destroying-threads
That is the C# advice, run the code in a separate process and kill the process.
What we are saying is you can use a cancellation token and regularly check in the thread if it is time to end and then throw an exception to end the thread. This is supported and is cooperative. I have posted example code here of how to do this. The current problem is that in PowerShell there is no agreed per-thread cancellation token to check or any contract that PowerShell code has to to do this. What is not recommended is the hard termination of an unresponsive thread. |
Can you tell us what these functions are that you are referring to so that we are talking about the same thing? A URL to the Microsoft documentation would be good. |
Lose some, even most, and then continue. You cannot throw away the entire apple just because there are wormholes on it. |
|
@kasini3000 Thanks for that. I wrote a simple test where the thread writes to a file in a simple loop and then is terminated. The output was What that shows is that the thread was stopped in a controlled manner because the finally was called while What was interesting was finally was called, but the catch was not. If you remove the So it cannot stop a job that is still running, however if the thread is still running when the main thread ends then it is presumably stopped with with If that is enough for you then you have your building blocks. |
|
What is you conclusion? can soldiers continue fight without 1 arm? |
Nobody is loosing any limbs or having any threads terminated! I wrote a custom I wrote a So far it confirms our understanding that no threads are terminated, all the job removal and attempts to stop long running Finally, if you leave the |
|
I have published the source to my simple |
;( Refer to Golang's approach ： WithTimeout |
|
I have found many stop thread powershell codes, which indicates that this is a common requirement. I will build my own application based on this. Coming soon. https://github.com/proxb/PInvoke/blob/master/Stop-Thread.ps1 https://github.com/hlldz/Phant0m/blob/master/old/Invoke-Phant0m.ps1#L1054 https://github.com/das-lab/mpsd/blob/main/powershell_benign_dataset/2760.ps1 |
|
All your links use the Win32 TerminateThread API so are neither recommended or portable. The Microsoft recommendation for Cancellation in Managed Threads is to use CancellationTokens exactly as we have been proposing here. I recommend to use threads for code that you trust and that use CancellationTokens or have implemented the Cmdlet.StopProcessing correctly. For all other code that you may need to cancel; run it in a separate process as recommended by Microsoft. Note that all external programs that you run, eg "ssh", "nslookup", "ping" etc are already running in separate processes so should not be a problem. |
|
The process can hang, so someone invented the kill command. |
First computers could only run on program at a time. Later computers supported processes to varying levels of degree to allow multiple programs to run on the expensive hardware. If you do not have pre-emptive multi-tasking and memory protection then all other processes are at risk from the worst program. Compare Windows 3.1 to OS/2, or Windows 95 to Windows NT, the system with process and kernel isolation and strong memory protection provides the most reliable system. Consider Windows 3.1, Windows 95 and any Macintosh System below 10 were absolutely terrible in terms of process isolation. Programming errors could cause memory corruption in any process. The game changer in reliability of systems was the introduction of virtual memory, preemptive multitasking and memory protection. So as you mentioned the Now if you look at how UNIX shells operate, they don't need to use threads to achieve efficient pipelines and parallelism. They work using fork and copy-on-write, so rather than creating a thread the entire process is cloned and two versions of the same program run using the same memory but with memory protection between them. Only on memory-write is new memory allocated to manage the difference in memory state on a page by page basis. Either could terminate and the other would be unaffected, except for the SIGCHLD signal going to the parent. Threading in a single process is much harder. It is hard to write robust preemptive multithreading code. It is hard when you write the entire program, it is even harder when you then include 3rd party code into your process which does not abide by the rules. So I will keep mentioning why The TerminateThread does not care if you are running PowerShell script, C# code or native C or C++ code. It will terminate it all without giving the code the opportunity to either protect itself or clean up. If you want to put this in perspective, this is an ideal way to corrupt an SQLite database. The updates to the database were taking too long so terminate the thread! You risk corrupting the structures used to manage the data and transaction control. In the POSIX world |
|
PowerShell is often compared to python, both high level scripting languages. Let's see what they have to say about it... https://stackoverflow.com/questions/323972/is-there-any-way-to-kill-a-thread
Exactly the same story.
Exactly what we are saying with cooperative threads. |
|
Another approach that you can do yourself is to create a cancellation token to define the time a thread should take and then use Register-ObjectEvent to register a handler to terminate the job. You need an actual dotnet class with an event that can be used by Register-ObjectEvent, this also registers an Action delegate with the cancellation token. Create your thread as normal Now set up the cancellation token to expire after 5000 milliseconds Set up the objects to connect the cancellation token to generate a PowerShell event for that job Now wait for the event and stop the associated job, the job identifier comes from the event And get the result of the terminated job So those are the building blocks, it shows you can manage both remote and local jobs with your own timeout mechanism. |
|
You could implement the stop-job within the action of the object event. I find you still need Wait-Event to be running to process the event queue. The New-Event is to unblock the waiting Wait-Event. |
|
Have updated the above package to let you cancel a |
|
@kasini3000 example of using a cancellation token to implement a thread timeout Result is Multiple threads can share the same cancellation token, so create a single cancellation token to be cancelled after 5 minutes and start twenty threads using the same cancellation token, then all will be cancelled in 5 minutes. With the PSGallery module I published you can do that right now. |
Golang's approach is still cooperative. Go will still panic and stop the process if things are not in order. |
|
This example creates one cancellation token with cancels after 5000ms. The same token is passed to 6 threads, they are all cancelled at the same time Output is |
|
Is there any good solution to call a .net async method with cancellation? Nothing seams to work, beside to kill the pwsh instance. The reader will be later inside a $TypeDefinition = @'
using System;
using System.Threading;
public class HttpClientHandlers
{
public static CancellationTokenSource GetConsoleCts() {
var cts = new CancellationTokenSource();
System.ConsoleCancelEventHandler handler = null;
handler = (sender, e) => {
//e.Cancel = true;
cts.Cancel();
if(handler is not null) {
Console.CancelKeyPress -= handler;
handler = null;
}
};
Console.CancelKeyPress += handler;
return cts;
}
}
'@
Add-Type -TypeDefinition $TypeDefinition
$cts = [HttpClientHandlers]::GetConsoleCts()
#...
$reader = new-object System.IO.StreamReader($stream)
while(!$reader.EndOfStream) {
$line = $reader.ReadLineAsync($cts.Token).GetAwaiter().GetResult()
Write-Host $line
}
Write-Host "end" |
|
Unfortunately PowerShell is not really task/async/await aware. Within a cmdlet it relies on thread local storage for context. I would suggest that within a cmdlet you create a child thread (not with ThreadJob) run your async tasks from that and wait on an event in your cmdlet that you indicate all your async tasks are finished and then return. It isn't safe to call PSCmdlet.WriteObject for instance from within a task that has lost all connection with the thread that PowerShell called your cmdlet's methods on. An alternative would be to use rhubarb-geek-nz.CancellationTokenEvent/ and use Invoke-CommandWithCancellationToken to call Invoke-WebRequest, that would keep all your code in PowerShell. |
|
@rhubarb-geek-nz thx, for your help and input. Because the lack of support from Invoke-WebRequest for HttpRequest streaming and WebSockets. I am forced to use the dotnet Types & instances. I can query and execute commands to the Kubernetes API without any 3th party compiled assemblies from the pwsh 7.4 The issue persist that currently the pwsh Runspace/Host does not provide any native Signal/WaitHandle/Callback to abort an operation. To call sync-over-async method in a STA environment will always produce a lock, but it does not mean that the inside async-method can not be cancelled with over a Something like this would already useful in many await/async situations. $ct = $Host.Runspace.Aborted
$line = $reader.ReadLineAsync($ct).GetAwaiter().GetResult()or something like the new $lines = Invoke-Pipe {
$reader = $using:reader
$ct = $MyCancellation
while(!$reader.EndOfStream) {
$line = $reader.ReadLineAsync($MyCancellation).GetAwaiter().GetResult()
Write-Output $line
}
} |
Agreed, I suggested that Common Parameters should include a CancellationToken. Signals don't really fit with my idea of PowerShell, they are process owned, where as I see the point of PowerShell is to write small reusable scripts, modules, functions and cmdlets that do not own the process and are then assembled for some higher purpose to solve a problem. My theory is that the PowerShell host itself should have the signal handler and the root console interactive session or script gets a cancellation token that will be tripped by that. Good luck with your solution, sounds fun. I wrote an ASP web server that uses HttpContext.RequestAborted to call PowerShell.StopAsync when a response was no longer worthwhile as the client had disconnected. |
|
I found a method to get the SIGINT signal (windows and linux) from the pwsh.exe to be useable for sync-over-async calls on Runspace1. https://gist.github.com/Zetanova/9e1a0d5e35d9ce876840b9d9b22445b3 It can be used in dotnet async method calls to be able to cancel them like with the pipe StopProcessing behavior. |
Summary of the new feature / enhancement
Add 2 parameters to Start-ThreadJob: timeout seconds, priority
Proposed technical implementation details (optional)
priority:
A single asynchronous task has no timeout and no priority， It is very original.
The priority range is 1-9, and the default is 5,priority 1 will be executed first.
timeoutseconds:
For that things, I think thread-timeout is a good medicine:
Invoke-WebRequest not responding to «Ctrc+c» PowerShell#16145
The regularity of unpredictable execution timelike：
golang : “context.WithTimeout”
python : timeout* in threading lib
It is better to add a Boolean type of parameter : -AutoRemoveJob
whether to automatically remove threadjob after timeout,
parameter likes “receive-job -AutoRemoveJob”
In this way, users can only consider new thread and put ps1 script code.
both "ForEach-Object -Parallel" and "Start-ThreadJob" without timeout, This has seriously affected my development of "kasini3000" at present.
Powershell community do a "engine"，i do a "car". the car is kasini3000 and k4t .
I love powershell. I want to give my opinion to the engine.
I hope the engine factory can transform a better engine.
I think timeoutseconds can make the ps1 script in the thread more robust.
I hope powershell will become stronger and stronger.
The text was updated successfully, but these errors were encountered: