Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update SharedArrayBuffer + add service-worker attribute details #134

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Update SharedArrayBuffer + add service-worker attribute details #134

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
24f98a5
Update SharedArrayBuffer + add service-worker attribute details
WebReflection Jul 26, 2024
d5dc4f1
Edits.
ntoll Jul 31, 2024
94ec035
fixed typo within a Python comment around attributes
WebReflection Jul 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 28 additions & 47 deletions docs/faq.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ encounter.

### SharedArrayBuffer

This is the first and most common error users may encounter with PyScript.
This is the first and most common error users may encounter with PyScript:

!!! failure

Expand All @@ -58,52 +58,35 @@ This is the first and most common error users may encounter with PyScript.
you see this message:

```
Unable to use SharedArrayBuffer due insecure environment.
Please read requirements in MDN: ...
Unable to use `window` or `document` -> https://docs.pyscript.net/latest/faq/#sharedarraybuffer
```

The error contains
[a link to MDN](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer#security_requirements)
but it's the amount of content provided on this topic is overwhelming.

#### When

This error happens when **the server delivering your PyScript application is
incorrectly configured**. It fails to provide the correct headers to handle
security concerns for web workers, or you're not using
[mini-coi](https://github.com/WebReflection/mini-coi#readme) as an alternative
solution. (These requirements are explored
[in the worker page](../user-guide/workers#http-headers).)

**And** at least one of the following scenarios is true:

* There is a `worker` attribute in the *py* or *mpy* script element and the
[sync_main_only](https://pyscript.github.io/polyscript/#extra-config-features)
flag is not present or not `true`.
* There is a `<script type="py-editor">` that uses a *worker* behind the
scenes.
* There is an explicit `PyWorker` or `MPWorker` bootstrapping somewhere in your
code.

!!! info
This happens when you're unable to access objects in the main thread (`window`
and `document`) from code running in a web worker.

If `sync_main_only` is `true` then interactions between the main thread and
workers are limited to one way calls from the main thread to methods
exposed by workers.
This error happens because **the server delivering your PyScript application is
incorrectly configured** or **a `service-worker` attribute has not been used in
your `script` element**.

If `sync_main_only = true`, the following caveats apply:
Specifically, one of the following three problem situations applies to your
code:

* It is not possible to manipulate the DOM or do anything meaningful on the
main thread **from a worker**. This is because Atomics cannot guarantee
sync-like locks between a worker and the main thread.
* Only a worker's `pyscript.sync` methods are exposed, and they can only be
awaited from the main thread.
* The worker can only `await` main thread references one after the other, so
developer experience is degraded when one needs to interact with the
main thread.
* Because of the way your web server is configured, the browser limits the use
of a technology called "Atomics" (you don't need to know how it works, just
that it may be limited by the browser). If there is a `worker` attribute in
your `script` element, and your Python code uses the `window` or `document`
objects (that actually exist on the main thread), then the browser limitation
on Atomics will cause the failure, unless you reconfigure your server.
* There is a `<script type="py-editor">` (that must always use a worker behind
the scenes) and no fallback has been provided via a `service-worker`
attribute on that element.
* There is an explicit `PyWorker` or `MPWorker` instance **bootstrapping
somewhere in your code** and no `service_worker` fallback has been provided.

If your project simply bootstraps on the main thread, none of this is relevant
because no worker requires such special features.
All these cases have been documented with code examples and possible solutions
in our section on [web workers](../user-guide/workers/).

#### Why

Expand All @@ -122,21 +105,19 @@ CPU. It idles until the referenced index of the shared buffer changes,
effectively never blocking the main thread while still pausing its own
execution until the buffer's index is changed.

As overwhelming or complicated as this might sounds, these two fundamental
As overwhelming or complicated as this might sound, these two fundamental
primitives make main ↔ worker interoperability an absolute wonder in term of
developer experience. Therefore, we encourage folks to prefer using workers
over running Python in the main thread. This is especially so when using
Pyodide related projects, because of its heavier bootstrap or computation
requirements. Using workers ensures the main thread (and thus, the user
interface) remains unblocked.

Unfortunately, due to security concerns and potential attacks to shared
buffers, each server or page needs to allow extra security to prevent malicious
software to read or write into these buffers. But be assured that if you own
your code, your project, and you trust the modules or 3rd party code you need
and use, **there are less likely to be security concerns around this topic
within your project**. This situation is simply an unfortunate "*one rule catch
all*" standard any server can either enable or disable as it pleases.
Unfortunately, we can patch, polyfill, or workaround, these primitives but
we cannot change their intrinsic nature and limitations defined by web
standards. However, there are various solutions for working around such
limitations. Please read our [web workers](..//user-guide/workers/)
section to learn more.

### Borrowed proxy

Expand Down
9 changes: 7 additions & 2 deletions docs/user-guide/first-steps.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,13 +80,18 @@ attributes:
* `worker` - a flag to indicate your Python code is to be run on a
[web worker](workers.md) instead of the "main thread" that looks after the user
interface.
* `target` - The id or selector of the element where calls to
* `target` - the id or selector of the element where calls to
[`display()`](../../api/#pyscriptdisplay) should write their values.
* `terminal` - A traditional [terminal](terminal.md) is shown on the page.
* `terminal` - a traditional [terminal](terminal.md) is shown on the page.
As with conventional Python, `print` statements output here. **If the
`worker` flag is set the terminal becomes interactive** (e.g. use
the `input` statement to gather characters typed into the terminal by the
user).
* `service-worker` - an optional attribute that allows you to slot in a custom
[service worker](https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API)
(such as `mini-coi.js`) to fix header related problems that limit the use
of web workers. This rather technical requirement is fully explained in
the [section on web workers](../workers/#option-2-service-worker-attribute).

!!! warning

Expand Down
143 changes: 110 additions & 33 deletions docs/user-guide/workers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,28 +7,25 @@ unresponsive. **You should never block the main thread.**
Happily, PyScript makes it very easy to use workers and uses a feature recently
added to web standards called
[Atomics](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Atomics).
You don't need to know about Atomics to use web workers, but the underlying
[coincident library](architecture.md#coincident)
**You don't need to know about Atomics to use web workers**, but it's useful to
know that the underlying [coincident library](architecture.md#coincident)
uses it under the hood.

!!! info

Sometimes you only need to `await` in the main thread the result of a call
to a method exposed in a worker.
Sometimes you only need to `await` in the main thread on a method in a
worker when neither `window` nor `document` are referenced in the code
running on the worker.

In such a limited case, and on the understanding that **code in the worker
will not be able to reach back into the main thread**, you should
use the [`sync_main_only` flag](../configuration/#sync_main_only) in your
configuration.

While this eliminates the need for the Atomics related header configuration
(see below), the only possible use case is to **return a serialisable
result from the method called on the worker**.
In these cases, you don't need any special header or service worker
as long as **the method exposed from the worker returns a serializable
result**.

## HTTP headers

For Atomics to work **you must ensure your web server enables the following
headers** (this is the default behaviour for
To use the `window` and `document` objects from within a worker (i.e. use
synchronous Atomics) **you must ensure your web server enables the following
headers** (this is the default behavior for
[pyscript.com](https://pyscript.com)):

```
Expand All @@ -38,27 +35,107 @@ Cross-Origin-Embedder-Policy: require-corp
Cross-Origin-Resource-Policy: cross-origin
```

If you are not able to configure your server's headers, use the
[mini-coi](https://github.com/WebReflection/mini-coi#readme) project to
achieve the same end.

!!! Info

The simplest way to use mini-coi is to place the
[mini-coi.js](https://raw.githubusercontent.com/WebReflection/mini-coi/main/mini-coi.js)
file in the root of your website (i.e. `/`), and reference it as the first
child tag in the `<head>` of your HTML documents:

```html
<html>
<head>
<script src="/mini-coi.js" scope="./"></script>
<!-- etc -->
</head>
<!-- etc -->
</html>
If you're unable to configure your server's headers, you have two options:

1. Use the [mini-coi](https://github.com/WebReflection/mini-coi#readme) project
to enforce headers.
2. Use the `service-worker` attribute with the `script` element.

### Option 1: mini-coi

For performance reasons, this is the preferred option so Atomics works at
native speed.

The simplest way to use mini-coi is to copy the
[mini-coi.js](https://raw.githubusercontent.com/WebReflection/mini-coi/main/mini-coi.js)
file content and save it in the root of your website (i.e. `/`), and reference
it as the first child tag in the `<head>` of your HTML documents:

```html
<html>
<head>
<script src="/mini-coi.js"></script>
<!-- etc -->
</head>
<!-- etc -->
</html>
```

### Option 2: `service-worker` attribute

This allows you to slot in a custom
[service worker](https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API)
to handle requirements for synchronous operations.

Each `<script type="m/py">` or `<m/py-script>` may optionally have
a `service-worker` attribute pointing to a locally served file (the
same way `mini-coi.js` needs to be served).

* You can chose `mini-coi.js` itself or *any other custom service worker*,
as long as it provides either the right headers to enable synchronous
operations via Atomics, or it enables
[sabayon polyfill events](https://github.com/WebReflection/sabayon?tab=readme-ov-file#service-worker).
* Alternatively, you can copy and paste the
[sabayon Service Worker](https://raw.githubusercontent.com/WebReflection/sabayon/main/dist/sw.js)
into your local project and point at that in the attribute. This will
not change the original behavior of your project, it will not interfere with
all default or pre-defined headers your application uses already but it will
**fallback to a (slower but working) synchronous operation** that allows
both `window` and `document` access in your worker logic.

```html
<html>
<head>
<!-- PyScript link and script -->
</head>
<body>
<script type="py" service-worker="./sw.js" worker>
from pyscript import window, document

document.body.append("Hello PyScript!")
</script>
</body>
</html>
```

!!! warning

Using sabayon as the fallback for synchronous operations via Atomics
should be **the last solution to consider**. It is inevitably
slower than using native Atomics.

If you must use sabayon, always reduce the amount of synchronous
operations by caching references from the *main* thread.

```python
# ❌ THIS IS UNNECESSARILY SLOWER
from pyscript import document

# add a data-test="not ideal attribute"
document.body.dataset.test = "not ideal"
# read a data-test attribute
print(document.body.dataset.test)

# - - - - - - - - - - - - - - - - - - - - -

# ✔️ THIS IS FINE
from pyscript import document

# if needed elsewhere, reach it once
body = document.body
dataset = body.dataset

# add a data-test="not ideal attribute"
dataset.test = "not ideal"
# read a data-test attribute
print(dataset.test)
```

In latter example the number of operations has been reduced from six to just
four. The rule of thumb is: _if you ever need a DOM reference more than once,
cache it_. 👍


## Start working

To start your code in a worker, simply ensure the `<script>`, `<py-script>` or
Expand Down