Allow field-modules to use special HTML5 input-elements (e.g. date pickers)

[FlashSystems]

I tried to add a HTML5 date picker to a tiddler field. Using a field-module I was able to transform the displayed INPUT-element into a date picker:

title: $:/plugins/flashsystems/todo/duefield.js
type: application/javascript
module-type: tiddlerfield

module.exports = {
	name: "due",
	editType: "date"
};

But now I've got a problem. The HTML5 date picker can not use TiddlyWiki's date format as an input and TiddlyWiki can not use the output of the date picker as a date.

I've implemented a solution, but I'm not sure, if it is a good idea: I extended the field-module by two new methods: toValue and fromValue. They are called by the editor factory to convert the field-value into the value of the INPUT-Element and vice versa. If they're not defined, they are a no-op.

A field-module using this will look like that:

title: $:/plugins/flashsystems/todo/duefield.js
type: application/javascript
module-type: tiddlerfield

module.exports = {
	name: "due",
	editType: "date",
	toValue: function(twDateString) {
		let date = $tw.utils.parseDate(twDateString);
		return (date instanceof Date) && !isNaN(date.valueOf()) ? date.toISOString().substring(0, 10) : "";
	},
	fromValue: function(isoDateString) {
		let date = new Date(isoDateString);
		return (date instanceof Date) && !isNaN(date.valueOf()) ? $tw.utils.stringifyDate(date) : "";
	}
};

This new feature would allow field-modules to use all kinds of HTML5-INPUT-elements: Date pickers, phone-number input, time selectors, etc. Plugins can supply their own field-modules. TiddlyWiki only needs to be extended by approx. 15 Lines of code in editor/factory.js.

I can supply a pull request with the necessary changes. But before I do it, I'd like to have some feedback of the more experienced TiddlyWiki developers. Is this the right way to do it? Are there any side effects I'm not aware of? Is it worthwhile to supply such an interface?

[Jermolene]

Thanks @FlashSystems, this is great work, and shows an excellent knowledge of the core internals.

However, part of the context is that I now see the tiddler field modules as being a mistake. The problem is that they confer special behaviour on specific fields in a way that is impossible for end users to replicate. As it's turned out, for every special behaviour of the "color", "modified" or "tags" field, users want to be able to confer the same behaviour on their own custom fields. This became evident early on when users started asking about additional fields with the behaviour of the "tags" field.

As ever, backwards compatibility constrains us to retain tiddler field modules, but we now try to handle things through explicit type coercion and explicit templating.

So, it's not clear the best way to extend the edit-text widget to support HTML 5 date inputs.

One option would be to add generic support for date fields in the same format that the HTML 5 date input generates. It would be reasonably straightforward as there is a single $tw.utils.parseDate() method that would need extending. However, that approach would break existing user code that manipulates the YYMMDDHHMMSS... date value as a string.

Alternately, we could extend the edit text widget to recognise type=date and do explicit type coercion as needed, presumably via something similar to you toValue() and fromValue() methods.

[FlashSystems]

Hello @Jermolene, thank you for your detailed reply. I understand your doubts regarding the field modules.

But I hesitate hard-coding type coercion into the edit text widget. Maybe field modules have been a mistake, but from an extensibility and isolation standpoint I like the general idea.

What do you think about creating a new module type. Let's call them "type modules" that are supplying the toValue and fromValue methods. They are tied to the type of the edit element, not the field. The whole thing might look like this:

title: $:/plugins/flashsystems/todo/duefield.js
type: application/javascript
module-type: edittype

module.exports = {
	editType: "date",
	toValue: function(twDateString) {
		let date = $tw.utils.parseDate(twDateString);
		return (date instanceof Date) && !isNaN(date.valueOf()) ? date.toISOString().substring(0, 10) : "";
	},
	fromValue: function(isoDateString) {
		let date = new Date(isoDateString);
		return (date instanceof Date) && !isNaN(date.valueOf()) ? $tw.utils.stringifyDate(date) : "";
	}
};

The modules are not meant to be created or modified by the user. They supply the type coercion for different types of HTML5 input elements. That way it is easy to include new types of input boxes into TiddlyWiki as they become supported by the browsers, without having to modify the edit text widget. TiddlyWiki could just include "type modules" for each edit-type it wants to support. Even dropping support for an edit type that has gone out of favor would be possible and and well isolated.

As an added benefit, plugins could supply "type modules" for special input elements that are not common enough for TiddlyWiki to support out of the box.

[pmario]

What do you think about creating a new module type. Let's call them "type modules" that are supplying the toValue and fromValue methods. They are tied to the type of the edit element, not the field. The whole thing might look like this:

I do like this idea. .. From my point of view, it would be worth to explore it further.

[FlashSystems]

I'll create a pull request containing a proof of concept implementation for further discussion.

[Jermolene]

Thanks @FlashSystems.

What do you think about creating a new module type. Let's call them "type modules" that are supplying the toValue and fromValue methods. They are tied to the type of the edit element, not the field. The whole thing might look like this:

My only hesitation now is to consider whether we can implement more of the logic in wikitext/filters, so that it is more hackable.

For example, we could add a pair of filters to the edit widget that are used to convert the edit widget value to and from the associated tiddler value. We might need to extend the filters in order to do the ISO date to TW date mapping, but that's a useful capability to have in wikitext anyway.

Then we could have a set of defaults stored in config tiddlers. For example, $:/config/EditWidgetMapping/Get/date might contain something along the lines of:

[<currentTiddler>parsedate[ISO]format:date[YYMMDDHHmmSSsss]]

The reason for giving this consideration is that we are generally trying to lift more of the core logic out of JS and into JavaScript (for example, I'm working on a wikitext implementation of the navigator widget).

[saqimtiaz]

For example, we could add a pair of filters to the edit widget that are used to convert the edit widget value to and from the associated tiddler value. We might need to extend the filters in order to do the ISO date to TW date mapping, but that's a useful capability to have in wikitext anyway.

This is the approach that I have taken in my own work by subclassing the $edit-text widget and it works well in practice.

[FlashSystems]

Hello @Jermolene,

I'm not convinced that hack-ability is of much concern here. A "type module" will convert between two standardized data formats: TiddlyWiki's internal representation and the W3C standardized format of the input type. Wouldn't the user expect this to just work? If I specify type=date or type=datetime-local on an EditTextWidget I'd expect the edit widget (or field) to use TiddlyWikis internal format, not the ISO-8601 format.

Don't get me wrong. I absolutely love TiddlyWiki's hack-ability. It's one of its greatest strengths. And for "field modules" I think this would be a great alternative. Being able to define the type (and maybe a validation filter) for a field via a tiddler would be awesome. But regarding "type modules" I'd rather aim for "Just do what the user expects".
I think "Type modules" are for TiddlyWiki developers to accommodate changes in the W3C standard. Not for the user to modify TiddlyWiki's behavior.

There is one additional concern I have with using a filter: If some browser (now or in the future) implements one of the types slightly wrong (maybe not adhering exactly to ISO-8601 format) or if there is a change in the standard it is much easier to accommodate that in JavaScript and fix the problem. The user will only see TiddlyWiki's format as if nothing happened.

[Jermolene]

A few thoughts:

• I think there might be other use cases for the ability to control the mapping between an input control and the value of a tiddler. For example, something that has fairly often been asked for is the ability to edit a tiddler and its fields in a single text area, using the same format as .tid files. The obvious implementation pattern would be to introduce custom mappings between the text area value and the tiddler
• Having another set of JS modules that do yet another set of arbitrary string manipulations risks a lot of near duplication, making the implementation and docs more confusing. We already have field modules getters/setters, filter modules, view widget formats, JS macros, all of which are very similar
• Meanwhile, we've grown the filter language into a strong general purpose system that permits logic to be expressed that we haven't directly anticipated, and so reusing it gives us good extensibility
• The existing attributes that are concerned with data binding could be recast as shortcuts for common patterns

In situations like this it can seem frustrating to have to discuss possibilities and hypotheticals while a functioning PR sits there unmerged. You'll appreciate that the problem is that we only get one chance to get the core right because of our backwards compatibiliy policies. One thing that bears into the discussion is the practicality of making the functionality available as a plugin. In this case, I think it would actually be fairly easy to make new edit-date widget that inherits from the edit-text widget and overrides a couple of methods. I'm not suggesting that you should do that, just that the fact that it appears to be possible is one the factors tending towards caution before merging the PR.

[FlashSystems]

I've created pull request #6329 that contains an implementation of the proposed "type modules" and some documentation.

Your feedback would be greatly appreciated.

[Jermolene]

Thanks @FlashSystems much appreciated. I'll give specific feedback over at the PR, but will continue the thread above as well.

[FlashSystems]

Hello @Jermolene,

no worries. I'm not getting frustrated and really appreciate the time you take to discuss the pros and cons of my idea and explaining your vision of TiddlyWiki.

If I understand your first bullet point correctly, the current 1:1 relationship between a field and the corresponding edit-widget would be changed into a 1:n relationship. This approach would be incredibly flexible. But I think implementing such a change would be beyond my capabilities.

My goal was to make the HTML5 date picker usable with TiddlyWiki. I'd like to tackle this in the following way that I hope is more in line with your plans for TiddlyWiki:

• I'd use one of your first suggestions and implement a toValue and fromValue parameter for the EditTextWidget. These accept filter syntax and are used for transformation of TiddlyWiki's internal value into the value of the input tag.
• Add an errorClass parameter to the EditTextWidget. This value is used if the fromValue filter returns no value. More on this later.
• Add a filter for parsing ISO-dates.
• Extend the $:/core/ui/EditTemplate/fields tiddler to allow transclusion of a tiddler for editing the field. More on this later, too.

The errorClass parameter

I've experimented with the toValue/fromValue-filter approach a little bit. There is one problem: If the filter is used to parse a value that is entered by the user, this value may be invalid while it is entered. For example 2021- is an invalid date until it is extended to 2021-10-12. If I update the bound field value on every change, tiddlers using this value would show some irritating flickering, while the value changes from empty to a new value and back to empty again while I type 2021 (which is parsed as 2021-01-01), 2021- (which is invalid), 2021-1 (which is valid) and so on.
I thought it would be better to not update the bound field if the filter returns no value. But in this case, the user might no know, that the value is invalid and hitting the save-button will not update the field. That would be... surprising.
The errorClass parameter solves this problem. This CSS class is added to the input tags CSS classes as long as the fromValue filter is returning an empty set of values. That way the user knows, that this value is invalid and will not be saved.
This has the added benefit of allowing to implement a field validator. Just create a fromValue filter that returns the current value or (if the value is invalid) no value at all. The toValue filter will not be needed.

Extend the $:/core/ui/EditTemplate/fields tiddler

My task of making the HTML5 date picker usable, is still not completed. I can still not use a HTML5 date picker for the due-field of my tiddlers. To make this possible, the $:/core/ui/EditTemplate/fields tiddler is extended to transclude the tiddler named "$:/core/config/EditTemplateFields/Templates/" + currentField. If this tiddler does not exist, the existing EditTextWidget is used as a fallback. That way, the implementation is completely backwards compatible. The currentTiddler and currentField variables are available in the transcluded tiddler. This has the additional benefit, that not only the EditTextWidget can be used. The user can use a dropdown-list or even something completely crazy. As long as it can be put into a tiddler that updates the filed, it will be fine.

Next steps

Do you think this approach would have a change of getting merged? If that's the case I'll close my first PoC pull request and create a new one, prototyping this approach.

[FlashSystems]

I've created a PoC for my proposal and put up a demo TiddlyWiki where you can try it out. The changes are in the fieldtemplates branch of my fork of the TW5 repo. I'd appreciate your feedback. The changes are a little more complex than my previous proposal. @Jermolene your suggestion using the filter syntax for the conversion makes this approach much more flexible. If you think, that these changes have a chance to get merged I'll clean up the code to comply with the style guide and update the documentation.

[Jermolene]

Thanks @FlashSystems it would be best if you could make a draft PR with the changes so that they are easier to see and we can use the GitHub commenting features to discuss changes at the code level.

[FlashSystems]

Hello @Jermolene, as requested I've created pull-request #6377.

[pmario]

Just to let you all know:

There is a discussion going on at Talk TW that discusses naming-conventions about tiddler fields to make date/time formats more interchangeable between TW plugins.