Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Javascript Plugin API (Custom panels, column menu items with JS actions) #2052

Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
305a4de
feat: first cut at datasette JS api
hydrosquall Mar 24, 2023
ab9bc5f
test: register table plugin using a test API
hydrosquall Mar 24, 2023
b30e609
feat(datasette): proof of concept - create JS plugin for adding colum…
hydrosquall Apr 2, 2023
2993896
Update API to support passing in other types of column metadata, such…
hydrosquall Apr 2, 2023
d348dfe
refactor: rename the dropdown icon in table.js
hydrosquall Apr 2, 2023
50cdf9b
feat: make datasette manager responsible for filter-rows API
hydrosquall Apr 2, 2023
e6b9301
add simple DOM selector plugin hooks example
hydrosquall Apr 2, 2023
d82b80a
first cut at a panel manager API
hydrosquall Apr 2, 2023
c2e7218
move demo plugins into own directory
hydrosquall Apr 2, 2023
3d48754
tidy: remove stale functions / todos from datasette JS plugin proposal
hydrosquall Apr 2, 2023
2d92b93
refactor(datasette): rename getColumnHeaderItems => getColumnActions …
hydrosquall Apr 14, 2023
37d7e3f
prf: InitDatasette -> datasette_init for case sensitivity / using pre…
hydrosquall Apr 20, 2023
8f744e7
prf: prefer has() to get() for set membership
hydrosquall Apr 20, 2023
9b773c7
prf: Convert column metadata names in column actions
hydrosquall Apr 20, 2023
06b4829
prf: provide plugin hook even for tables with 0 rows
hydrosquall Apr 20, 2023
cf504fe
prf: add JS handling for situations where tables have no rows
hydrosquall Apr 20, 2023
0536f5d
prf: getColumnActions -> makeColumnActions for a more action oriented…
hydrosquall Apr 20, 2023
cef6740
prf: getAboveTablePanelConfigs -> makeAboveTablePanelConfigs for more…
hydrosquall Apr 20, 2023
92e0916
feat(datasette-manager): Sync manager version with python version
hydrosquall Jul 1, 2023
a134f91
dev: relocate example plugins file to a demos folder
hydrosquall Jul 1, 2023
eb1f408
fix: move example plugins to correct directory, dynamically set datas…
hydrosquall Jul 11, 2023
cf5a9df
fix: return early if the page doesn't have a table element to attach to
hydrosquall Jul 11, 2023
b0aca42
Tiny whitespace tweak
simonw Oct 12, 2023
a7e7942
Fixed typo in plugin example
simonw Oct 12, 2023
e4538e7
Fixed typos in comments
simonw Oct 12, 2023
8ae479c
Ran Prettier
simonw Oct 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
202 changes: 202 additions & 0 deletions datasette/static/datasette-manager.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
// Custom events for use with the native CustomEvent API
const DATASETTE_EVENTS = {
INIT: "InitDatasette", // returns datasette manager instance in evt.detail
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Maybe "DatasetteInit" instead? Or "datasette_init"? Not sure if there's a common convention for custom event names...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm flexible, I saw a bunch of opinions about this here:

https://stackoverflow.com/questions/19071579/javascript-dom-event-name-convention

It looks like it might be safe to err on the side of all-lowercase. datasette_init or datasette.init both have a reasonable look to them.

}

// Datasette "core" -> Methods/APIs that are foundational
// Plugins will have greater stability if they use the functional hooks- but if they do decide to hook into
// literal DOM selectors, they'll have an easier time using these addresses.
const DOM_SELECTORS = {
/** Should have one match */
jsonExportLink: ".export-links a[href*=json]",

/** Event listeners that go outside of the main table, e.g. existing scroll lisetner */
tableWrapper: ".table-wrapper",
table: "table.rows-and-columns",
aboveTablePanel: '.above-table-panel',

// These could have multiple matches
/** Used for selecting table headers. Use getColumnActions if you want to add menu items. */
tableHeaders: `table.rows-and-columns th`,

/** Used to add "where" clauses to query using direct manipulation */
filterRows: ".filter-row",
/** Used to show top available enum values for a column ("facets") */
facetResults: ".facet-results [data-column]",
};


/**
* Monolith class for interacting with Datasette JS API
* Imported with DEFER, runs after main document parsed
*/
const datasetteManager = {
VERSION: 'TODO_INJECT_VERSION_OR_ENDPOINT_FROM_SERVER_OR_AT_BUILD_TIME',


// TODO: Should order of registration matter?

// Should plugins be allowed to clobber others or is it last-in takes priority?
// Does pluginMetadata need to be serializable, or can we let it be stateful / have functions?
plugins: new Map(),

registerPlugin: (name, pluginMetadata) => {
if (datasetteManager.plugins.get(name)) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: .has() over .get()

console.warn(`Warning -> plugin ${name} was redefined`);
}
datasetteManager.plugins.set(name, pluginMetadata);

// If the plugin partipates in the panel... update the panel.
if (pluginMetadata.getAboveTablePanelConfigs) {
datasetteManager.renderAboveTablePanel();
}
},

/**
* New DOM elements created each time the button is clicked so the data is not stale.
* Items
* - must provide label (text)
* - might provide href (string) or an onclick ((evt) => void)
*
* columnMeta is metadata stored on the column header (TH) as a DOMStringMap
* - column: string
* - columnNotNull: 0 or 1
* - columnType: sqlite datatype enum (text, number, etc)
* - isPk: 0 or 1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could columnNotNull and isPk be booleans instead? Also, they appear to be strings right now ("0" and "1" rather than 0 and 1, at least according to the "Log column metadata to console" example

Also, maybe column -> columnName? or just name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that makes sense, right now I'm passing through the data unparsed directly from the DOMStringMap.

Probably more ergonomic to use these precise datatypes and more opinionated field names, great suggestions - I'll apply these soon.

*/
getColumnActions: (columnMeta) => {
let items = [];
// Accept function that returns list of items with keys
// Required: label (text)
// Optional: onClick or href
datasetteManager.plugins.forEach(plugin => {
if (plugin.getColumnActions) {
// Plugins can provide multiple items if they want
// If multiple try to create entry with same label, the last one deletes the others
items.push(...plugin.getColumnActions(columnMeta));
}
});
// TODO: Validate item configs and give informative error message if missing keys.
return items;
},

/**
* In MVP, each plugin can only have 1 instance.
* In future, panels could be repeated. We omit that for now since so many plugins depend on
* shared URL state, so having multiple instances of plugin at same time is problematic.
* Currently, we never destroy any panels, we just hide them.
*
* TODO: nicer panel css, show panel selection state.
* TODO: does this hook need to take any arguments?
*/
renderAboveTablePanel: () => {

const aboveTablePanel = document.querySelector(DOM_SELECTORS.aboveTablePanel);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This returns null on tables that have no rows: ex in fixtures on http://localhost:8001/fixtures/123_starts_with_digits , I get:

caught TypeError: Cannot read properties of null (reading 'querySelector')
    at Object.renderAboveTablePanel (datasette-manager.js:95:50)
    at Object.registerPlugin (datasette-manager.js:51:24)
    at addPlugins (table-example-plugins.js:34:11)
    at HTMLDocument.<anonymous> (table-example-plugins.js:8:3)
    at initializeDatasette (datasette-manager.js:193:12)
    at HTMLDocument.<anonymous> (datasette-manager.js:201:3)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting- in those cases, do you think it still makes sense for the plugin to attempt to display (in which case there should be a fallback mount point, or the mount point should always show up regardless of whether there is data), or should a panel like this only show up for views that have some rows?

I think it's the first case, but wanted to check what you think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decided to always provide a hook so this won't be an issue

06b4829

but also added a bit of JS safety in case the template structure changes in the future.

cf504fe

let aboveTablePanelWrapper = aboveTablePanel.querySelector('.panels');

// First render: create wrappers. Otherwise, reuse previous.
if (!aboveTablePanelWrapper) {
aboveTablePanelWrapper = document.createElement('div');
aboveTablePanelWrapper.classList.add('tab-contents');
const panelNav = document.createElement('div');
panelNav.classList.add('tab-controls');

// Temporary: css for minimal amount of breathing room.
panelNav.style.display = 'flex';
panelNav.style.gap = '8px';
panelNav.style.marginTop = '4px';
panelNav.style.marginBottom = '20px';

aboveTablePanel.appendChild(panelNav);
aboveTablePanel.appendChild(aboveTablePanelWrapper);
}

datasetteManager.plugins.forEach((plugin, pluginName) => {
const { getAboveTablePanelConfigs: getPanelConfigs } = plugin;

if (getPanelConfigs) {
const controls = aboveTablePanel.querySelector('.tab-controls');
const contents = aboveTablePanel.querySelector('.tab-contents');

// Each plugin can make multiple panels
const configs = getPanelConfigs();

configs.forEach((config, i) => {
const nodeContentId = `${pluginName}_${config.id}_panel-content`;

// quit if we've already registered this plugin
// TODO: look into whether plugins should be allowed to ask
// parent to re-render, or if they should manage that internally.
if (document.getElementById(nodeContentId)) {
return;
}

// Add tab control button
const pluginControl = document.createElement('button');
pluginControl.textContent = config.label;
pluginControl.onclick = () => {

contents.childNodes.forEach(node => {
if (node.id === nodeContentId) {
node.style.display = 'block'
} else {
node.style.display = 'none'
}
});
}
controls.appendChild(pluginControl);

// Add plugin content area
const pluginNode = document.createElement('div');
pluginNode.id = nodeContentId;
config.render(pluginNode);
pluginNode.style.display = 'none'; // Default to hidden unless you're ifrst

contents.appendChild(pluginNode);
});

// Let first node be selected by default
if (contents.childNodes.length) {
contents.childNodes[0].style.display = 'block';
}
}
});
},


/** Selectors for document (DOM) elements. Store identifier instead of immediate references in case they haven't loaded when Manager starts. */
selectors: DOM_SELECTORS,

// Future API ideas
// Fetch page's data in array, and cache so plugins could reuse it
// Provide knowledge of what datasette JS or server-side via traditional console autocomplete
// State helpers: URL params https://github.com/simonw/datasette/issues/1144 and localstorage
// UI Hooks: command + k, tab manager hook
// Should we notify plugins that have dependencies
// when all dependencies were fulfilled? (leaflet, codemirror, etc)
// https://github.com/simonw/datasette-leaflet -> this way
// multiple plugins can all request the same copy of leaflet.
};


const initializeDatasette = () => {
// Hide the global behind __ prefix. Ideally they should be listening for the
// DATASETTE_EVENTS.INIT event to avoid the habit of reading from the window.

window.__DATASETTE__ = datasetteManager;
console.debug("Datasette Manager Created!");
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was worried that there might be browsers in which this would cause an error (because console.debug might not be defined), but as far as I can tell this has been supported in every modern browser for years at this point: https://console.spec.whatwg.org/ and https://developer.mozilla.org/en-US/docs/Web/API/console#browser_compatibility - so this is fine.


const initDatasetteEvent = new CustomEvent(DATASETTE_EVENTS.INIT, {
detail: datasetteManager
});

document.dispatchEvent(initDatasetteEvent)
}

/**
* Main function
* Fires AFTER the document has been parsed
*/
document.addEventListener("DOMContentLoaded", function () {
initializeDatasette();
});
100 changes: 100 additions & 0 deletions datasette/static/table-example-plugins.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
/**
* Example usage of Datasette JS Manager API
*/

document.addEventListener("InitDatasette", function (evt) {
const { detail: manager } = evt;
// === Demo plugins: remove before merge===
addPlugins(manager);
});

/**
* Examples for to test datasette JS api
*/
const addPlugins = (manager) => {

manager.registerPlugin("column-name-plugin", {
version: 0.1,
getColumnActions: (columnMeta) => {
const { column } = columnMeta;

return [
{
label: "Copy name to clipboard",
onClick: (evt) => copyToClipboard(column),
},
{
label: "Log column metadata to console",
onClick: (evt) => console.log(column),
},
];
},
});

manager.registerPlugin("panel-plugin-graphs", {
version: 0.1,
getAboveTablePanelConfigs: () => {
return [
{
id: 'first-panel',
label: "First",
render: node => {
const description = document.createElement('p');
description.innerText = 'Hello world';
node.appendChild(description);
}
},
{
id: 'second-panel',
label: "Second",
render: node => {
const iframe = document.createElement('iframe');
iframe.src = "https://observablehq.com/embed/@d3/sortable-bar-chart?cell=viewof+order&cell=chart";
iframe.width = 800;
iframe.height = 635;
iframe.frmaeborder = '0';
node.appendChild(iframe);
}
},
];
},
});

manager.registerPlugin("panel-plugin-maps", {
version: 0.1,
getAboveTablePanelConfigs: () => {
return [
{
// ID only has to be unique within a plugin, manager namespaces for you
id: 'first-map-panel',
label: "Map plugin",
// datasette-vega, leafleft can provide a "render" function
render: node => node.innerHTML = "Here sits a map",
},
{
id: 'second-panel',
label: "Image plugin",
render: node => {
const img = document.createElement('img');
img.src = 'https://datasette.io/static/datasette-logo.svg'
node.appendChild(img);
},
}
];
},
});

// Future: dispatch message to some other part of the page with CustomEvent API
// Could use to drive filter/sort query builder actions without page refresh.
}



async function copyToClipboard(str) {
try {
await navigator.clipboard.writeText(str);
} catch (err) {
/** Rejected - text failed to copy to the clipboard. Browsers didn't give permission */
console.error('Failed to copy: ', err);
}
}
Loading