[BUG] There is no way to ensure that rich.progress.Progress can be implemented at the same time as stream text output.

[DuskXi]

• I've checked docs and closed issues for possible solutions.
• I can't find my issue in the FAQ.

Describe the bug

I hope to use rich.progress.Progress and real-time streaming output without line breaks at the same time. At the same time, I hope that the text will not be modified by any subsequent operations after the line break appears.
I tried many ways but none of them worked.
I tried the following methods

Using Progress() + print(text, end="")
will make it invisible during the run until the process ends. This completely loses the effect of streaming

Using Progress() [main thread] + print(text, end="") [child thread]
will cause end="" to fail, and each print will wrap, which also seriously destroys the effect and format of streaming

Using Progress() + rpint(text, end="") or console.print(text, end="")
will cause the text in the same line to overwite the previously printed text.

Using Progress() + Live(auto_refresh=False) + rpint(text, end="")
Only the first line of streaming works, and each subsequent refresh will cause the previously output content to disappear.
Enabling the transient parameter is also useless

Platform

Windows

Click to expand

What platform (Win/Linux/Mac) are you running on? What terminal software are you using?

I may ask you to copy and paste the output of the following commands. It may save some time if you do it now.

If you're using Rich in a terminal:

python -m rich.diagnose
pip freeze | grep rich

If you're using Rich in a Jupyter Notebook, run the following snippet in a cell
and paste the output in your bug report.

from rich.diagnose import report
report()

╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── <class 'rich.console.Console'> ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ A high level console interface. │
│ │
│ ╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │
│ │ │ │
│ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ color_system = 'truecolor' │
│ encoding = 'utf-8' │
│ file = <_io.TextIOWrapper name='' mode='w' encoding='utf-8'> │
│ height = 22 │
│ is_alt_screen = False │
│ is_dumb_terminal = False │
│ is_interactive = True │
│ is_jupyter = False │
│ is_terminal = True │
│ legacy_windows = False │
│ no_color = False │
│ options = ConsoleOptions(size=ConsoleDimensions(width=374, height=22), legacy_windows=False, min_width=1, max_width=374, is_terminal=True, encoding='utf-8', max_height=22, justify=None, overflow=None, no_wrap=False, highlight=None, markup=None, height=None) │
│ quiet = False │
│ record = False │
│ safe_box = True │
│ size = ConsoleDimensions(width=374, height=22) │
│ soft_wrap = False │
│ stderr = False │
│ style = None │
│ tab_size = 8 │
│ width = 374 │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭── <class 'rich._windows.WindowsConsoleFeatures'> ───╮
│ Windows features available. │
│ │
│ ╭─────────────────────────────────────────────────╮ │
│ │ WindowsConsoleFeatures(vt=True, truecolor=True) │ │
│ ╰─────────────────────────────────────────────────╯ │
│ │
│ truecolor = True │
│ vt = True │
╰─────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────────────── Environment Variables ───────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ {'TERM': None, 'COLORTERM': None, 'CLICOLOR': None, 'NO_COLOR': None, 'TERM_PROGRAM': None, 'COLUMNS': None, 'LINES': None, 'JUPYTER_COLUMNS': None, 'JUPYTER_LINES': None, 'JPY_PARENT_PID': None, 'VSCODE_VERBOSE_LOGGING': None} │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
platform="Windows"

rich==13.9.2

[github-actions]

Thank you for your issue. Give us a little time to review it.

PS. You might want to check the FAQ if you haven't done so already.

This is an automated reply, generated by FAQtory

[willmcgugan]

Without code, I can't say what you are doing wrong. Please share a short piece of code that demonstrates the problem.

[DuskXi]

Thank you very much for your reply.
Below is the test I did, and the comments following each function are the console output status I intercepted

import threading
import time

from rich.live import Live
from rich.progress import Progress
from rich.console import Console
from rich import print as rprint


def test_progress_1():
    with Progress() as progress:
        task = progress.add_task("[Red]Running", total=100)
        for i in range(100):
            for j in range(100):
                print(f"{j}", end="")
                time.sleep(0.002)
            progress.update(task, advance=1)
            time.sleep(0.1)


# running:
# python3 test.py
# [Red]Running ━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━  44% 0:00:18
# after exit:
# 0123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384
# ......
# [Red]Running ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00

def test_progress_2():
    def print_thread():
        for j in range(100):
            print(f"{j}", end="")
            time.sleep(0.002)

    with Progress() as progress:
        task = progress.add_task("[Red]Running", total=100)
        for i in range(100):
            thread = threading.Thread(target=print_thread)
            thread.start()
            while thread.is_alive():
                time.sleep(0.1)
            thread.join()
            progress.update(task, advance=1)
            time.sleep(0.1)


# running:
# python3 test.py
# [Red]Running ━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━  44% 0:00:18
# after exit:
# 0123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384
# ......
# [Red]Running ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00


def test_progress_3():
    with Progress() as progress:
        task = progress.add_task("[Red]Running", total=100)
        for i in range(100):
            for j in range(100):
                rprint(f"{j}", end="")
                time.sleep(0.002)
            progress.update(task, advance=1)
            time.sleep(0.1)

# running:
# python3 test.py
# 68[Red]Running ━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  13% 0:00:32
# after exit:
# [Red]Running ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00


def test_progress_4():
    console = Console()
    with Progress(console=console) as progress:
        task = progress.add_task("[Red]Running", total=100)
        for i in range(100):
            for j in range(100):
                console.print(f"{j}", end="")
                time.sleep(0.002)
            progress.update(task, advance=1)
            time.sleep(0.1)

# running:
# python3 test.py
# 68[Red]Running ━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  13% 0:00:32
# after exit:
# [Red]Running ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00

def test_progress_5():

    console = Console()
    progress = Progress(console=console)
    task = progress.add_task("Downloading", total=100)

    with Live(console=console, auto_refresh=False, transient=True) as live:
        for i in range(100):
            for j in range(100):
                console.print(f"{j}", end="")
                time.sleep(0.002)
            progress.update(task, advance=1)
            live.update(progress) 
            live.refresh()
            time.sleep(0.1)

# running:
# python3 test.py
# 68[Red]Running ━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  13% 0:00:32
# after exit:
# [Red]Running ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00


def test_progress_6():
    console = Console()
    progress = Progress(console=console)
    task = progress.add_task("Downloading", total=100)

    with Live(console=console, auto_refresh=False, transient=True) as live:
        for i in range(100):
            for j in range(100):
                rprint(f"{j}", end="")
                time.sleep(0.002)
            progress.update(task, advance=1)
            live.update(progress) 
            live.refresh()  
            time.sleep(0.1)

# running:
# python3 test.py
# 0123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899Downloading ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   1% -:--:--01234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636
# Downloading ━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  19% 0:00:2901234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586
# exit:
# 0123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899Downloading ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   1% -:--:--01234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636

Without code, I can't say what you are doing wrong. Please share a short piece of code that demonstrates the problem.

[arnimarj]

I've hit a similar use case. I was running concurrent downloads and wanted a progress meter for each download, along with a log output when each (potentially failed) download finished.

Since both rich and the logging framework were using stdout, the outputs would sometimes mix. I solved it by guarding each relevant logging call with with progress.live._lock: ... for my instance of rich.progress.Progress.

But that's an undocumented feature I guess.

[peterjc]

I think I hit the same issue, sample output from an example based on that in snakemake/snakemake#3151 but modified so the background commands print to stdout and stderr:

$ ./snakemake_progress_bar_demo.py
Snakemake... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% -:--:-- 0:00:00INFO: Starting checksum for NC_011916.fna
INFO: Starting checksum for NC_010338.fna
INFO: Starting checksum for NC_014100.fna
INFO: Starting checksum for NC_002696.fna
Snakemake... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% -:--:-- 0:00:04DEBUG: Nearly finished NC_002696.fna
Done: NC_002696.fna.md5
Snakemake... ━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  25% -:--:-- 0:00:05DEBUG: Nearly finished NC_014100.fna
Done: NC_014100.fna.md5
Snakemake... ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━  50% 0:00:03 0:00:07DEBUG: Nearly finished NC_010338.fna
Done: NC_010338.fna.md5
Snakemake... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━  75% 0:00:02 0:00:08DEBUG: Nearly finished NC_011916.fna
Done: NC_011916.fna.md5
Snakemake... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 0:00:08
Snakemake return code 0

Note that output is often directly after the elapsed time. It doesn't seem to matter that the INFO: ... lines are to stdout, and DEBUG: ... were to stderr.

I guess this would be better if rich could leave the cursor at the first column (via CR), rather than at the end of its output?