Colors and buttons, oh my

[frosencrantz]

@saulpw you are a sly one. You taught VisiData how to handle buttons and how to do per character colors. 🤯 I wasn’t expecting that…

This works in cell values, column names, and the sidebar. I haven’t checked what it can do in the status bar.

For anyone else who missed this you can now use tags for coloring values and for making buttons like this one: [:red]tomatoes[:][:green]frog[:][onclick long-command input]button[:]

Commits are only on the develop branch:

• 3c0571e
• 0f8e6cf

[saulpw]

I haven't announced it broadly yet because I'm not entirely committed to the [:color] internal syntax. But yes it should work in most places!

[saulpw]

Also note that the color can be either a literal color string like red or 23 bold on yellow, or a color option, like warning (which will use the color_warning).

[geekscrapy]

Nice!! This is great news!

Might I suggest a common format for these codes? ANSI or even maybe those that are used in rich (https://rich.readthedocs.io/en/stable/markup.html#syntax)?

[saulpw]

[color] isn't specific enough for my taste, and we're not going to support [/bold] right now; we can't subtract color yet. ANSI codes might nicely interact with things like rich output, but they aren't easily inputtable, and we'd have to make up non-standard codes to indicate color options. What else could we model it after?

[frosencrantz]

I like @geekscrapy's suggestion of using the rich markup in that it would be consistent with another tool.

One thing I noticed when using the markup in cell values, is that the markup is part of the value. This causes anything that looks at the value of the cells sorting, regex, python expressions, etc may have issues because of the formatting. Another option might be to separate the markup from the content, or have a way to filter the value so it doesn't interfere with the values.

[saulpw]

I like @geekscrapy's suggestion too; I just don't think we can use bbcode, HTML, Markdown, or ANSI. Also note that the VisiData internal format is parsed linearly, and not nestable, which excludes many/most popular markup formats. Please suggest other alternatives!

[saulpw]

One thing I noticed when using the markup in cell values, is that the markup is part of the value.

Currently we have:

• getValue: the base value
• getTypedValue: used for most things, including aggregation and saving to typed formats
• getDisplayValue: the formatted value, used for grouping and saving to text formats

I guess for this purpose we would make a new function that gets the marked-up version. We maybe could rename getDisplayValue to getFormattedValue or getTextValue, and then add getDisplayText to include markup. I guess getDisplayText would then be used when passed to clipdraw, and getTextValue when passed to any computation or saving function.

This would make it possible to add non-computational markup via a special Column type (or a hacked-up plugin), and any markup in the value itself would be included in the value for computation and saving properly, but then that markup would also affect the cell display, which is may not be desirable. This would also mean that the fmtstr itself couldn't introduce "invisible" markup, though why you'd want that is not immediately clear to me.

But is this actually that useful? I've debated back-and-forth about whether the cell display should parse this internal markup or not, and/or how people might use it for the "conditional formatting" everyone loves so much. The first use case we have is to show positive numbers in green and negative numbers in red, and of course have that not affect any computation or saving, but also be applicable to existing columns. There is the formatter Column attribute that was introduced somewhat recently, which is all about using the fmtstr attribute to produce the "text value". Maybe we need a displayer attribute similarly to specify which function to use to generate the "display text"? By default I think it would just return the text/formatted value directly, with some method to indicate to not parse the markup; but then it could be changed to anything arbitrary and include markup.

This is a pretty major internal architectural change, btw. I'm tempted to just always display cell values literally (without parsing markup) until we have a solid solution to this, in large part because I don't see any use for non-conditional formatting within the value itself.

[frosencrantz]

I have not used this other than to confirm it works and a couple simple things. But giving VisiData the ability to add per character colors is useful. A couple things I would like to see/try:

• The ability to highlight a sorted column of values that have many common prefixes so that it is easier to see when multiple rows have the same prefix. I'm thinking about filepaths or urls here.
• Highlighting the matching part of a search.

I'm sure there are many more.
On the button side, it would be very useful to have per column, row or cell buttons.

When doing the computations it seems like computations would rather see the plain text value without all the markup. The displayer sounds like it could help with these cases too, and maybe make it easier to do some of the cases I've thought of so far. I do hope you keep these abilities, I'm not firmly tied to any solution yet. However, any solution should have a way for computations to ignore any markup. To me this seems important, based on my very simple usage so far.

[geekscrapy]

I totally agree with @frosencrantz here about displaying the markup. I would go a step further here and say that I users shouldn't really have exposure to markup. I'm thinking it should only really be used by loaders, plugins, [column] types and display formats. Users only really have the opportunity to turn it on and off.

Also, an example of the benefit of common format use would be that I use the "open in PAGER" (and a plugin that allows "open CELL in PAGER" by @ajkerrigan) and it would be really nice to provide the colouring to these plugins.

[geekscrapy]

If I have a formatter which inserts these codes (like below), should the colors be shown when the formatter is applied?

vdtype(color_str, 'c', formatter=lambda _,v: v.replace('x', '[:red]x[:]'))

Appreciate it's early days with the colorising, but I have a use case to highlight certain sections of a cell and thought I'd try it, but it doesn't seem to work for me

[saulpw]

We decided that cell values by default shouldn't colorize. If you want this, we'll have to create a Column.displayer_colorized (or something) that parses the cell into color/text pairs, and set the displayer attribute on the Column to "colorized".

[frosencrantz]

How would the Column.displayer_colorized be used? Buttons with onclick will also go away?

It sounds like it will still be possible to use this markup, but there will be an extra step to turn it on. Is that what you are thinking?

I would like to see a way for this new feature to still exist in some form.

[saulpw]

Yes, that's right, it would be disabled by default for cells, but still be accessible for some/all columns as the user or app has configured. Clicks/buttons would be the same (disabled by default, but available via this same mechanism). The nice thing is, that it should be straightforward to use any kind of markup that can be readily parsed into the color/attribute/onclick strings, without having to support all the different desired markups internally.