Skip to content

Commit

Permalink
IBX-8963: AI Actions (#315)
Browse files Browse the repository at this point in the history
* IBX-8689: AI Actions

* Add ct article to navigation, fix small issues

* Update AI doc following changes in the UI
  • Loading branch information
dabrt authored Nov 18, 2024
1 parent 596f03f commit a38a3b4
Show file tree
Hide file tree
Showing 15 changed files with 272 additions and 46 deletions.
17 changes: 17 additions & 0 deletions docs/ai_actions/ai_actions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
description: Learn about generative AI and its capabilities.
---

# AI Actions

AI Actions is a set of capabilities that enhance the efficiency and creativity of content editors.
The feature is not included in [[= product_name =]] by default.
It's an [LTS Update]([[= developer_doc =]]/release_notes/ibexa_dxp_v4.6/#lts-updates) that requires separate [installation and system configuration]([[= developer_doc =]]/ai_actions/install_ai_actions/).

Once AI Actions are installed and configured, content editors with `Action configuration/View` and `Action configuration/Execute` permissions have access to AI-driven tools.
For example, they can [enhance or improve the text](create_edit_content_items.md#ai-assistant) that they have written, or [generate alternative text](upload_images.md#ai) for images.

If you have the right permissions, you can decide what capabilities are available to the editors by enabling AI actions.
You can also modify the parameters of individual AI actions, to control the creativity of the AI or the cost involved with using it.

For more information, see [Work with AI actions](work_with_ai_actions.md#edit-existing-ai-actions).
Binary file added docs/ai_actions/img/ai_action_details.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/ai_actions/img/ai_action_options.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/ai_actions/img/ai_actions_list.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/ai_actions/img/ai_assistant.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
102 changes: 102 additions & 0 deletions docs/ai_actions/work_with_ai_actions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
---
description: Create new AI actions or modify existing ones to work faster and increase creativity.
---

# Work with AI actions

AI actions define what results are available to editors in AI-enabled areas, such as, for example, the AI Assistant.
If the AI Actions LTS update has been installed and configured in your application, and you have the required [permissions]([[= developer_doc =]]/permissions/policies/#ai-actions), including `Action configuration/Edit` and `Action configuration/Create`, you can reconfigure the existing AI actions, and create new ones.

## View AI actions

With the right permissions, you can view all AI actions configured in the application by navigating to the Admin Panel and selecting **AI actions**.

![AI actions in Admin Panel](img/ai_actions_list.png)

You can narrow down the list of AI actions by filtering it by the status, either Enabled or Disabled, or by the type.
Out of the box, there are two categories of AI Actions present in the system:

- **Text to text** - used by default in [online editor](create_edit_content_items.md#ai-assistant) for refining text, for example: "Rewrite text in formal tone"
- **Image to text** - used by default in the [image asset editing screen](upload_images.md#ai) to generate alternative text, for example: "Generate short alternative description of an image"

It may happen that a set of sample AI actions has been [installed with the AI Actions package]([[= developer_doc =]]/ai_actions/install_ai_actions/#install-sample-ai-action-configurations-optional), and there is a number of existing AI actions that you can modify and clone.

!!! note "Custom action types"

In your specific case, the types available can be different, and your organization's development team can create custom AI action types.
For more information, see [developer documentation]([[= developer_doc =]]/ai_actions/).

### View AI action details

Navigate to the Admin Panel and select **AI actions**.
In the **AI actions** list, click the name of an AI action to review its details.
For example, in the **Properties** tab, you can see specific settings that modify the prompt that is sent to an AI service.

![AI action details](img/ai_action_details.png)

## Edit existing AI actions

You can modify the existing AI actions.

1\. Navigate to the Admin Panel and select **AI actions**.

2\. In the **AI actions** list, click the **Edit** icon next to a name of the AI action that you want to modify.

3\. In the **Global properties** section, you can change the name and description of the AI action. You can also toggle the availability of the AI action between disabled and enabled.

4\. In the **Settings** area, change the settings that modify the behavior of an AI service that executes an AI action, for example:

- **Prompt** - modifies the default request by passing a verbal command, for example, "Make it short and formal."

!!! note "Default request"

The default request can be seen at the top of the settings area, on a light blue background.

- **Max tokens** - sets a maximum number of "[words](https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them)" or tokens that can be used in a single request by both the request and the response

- **Length of prompt output** - sets a maximum number of words of the generated result

- **Temperature** - controls the randomness of the response.
Takes a value between 0 and 1.
The higher the temperature, the more creative is the output

![AI action options](img/ai_action_options.png)

!!! note "Action settings availability"

Action settings differ depending on the AI service used, model implementation and actual action type.
Therefore the settings visible in your installation may vary from the ones presented above.

5\. Click **Save and close** to apply the changes or **Discard** to discard them and close the window.

## Create new AI actions

You can create AI actions that perform actions of different types, using different models, or action handlers.

!!! note "AI action models"

Before you can work with AI actions, models must be configured and enabled by your organization's development team.
If there are more AI service connectors available, you might be able to create AI actions that perform the same type of actions but use different models.
For more information, see [developer documentation]([[= developer_doc =]]/ai_actions/ai_actions_guide/#model).

1. Navigate to the Admin Panel and select **AI actions**.

1. In the **AI actions** list, click **Create**.

1. In the slide-out pane, make initial choices in the following fields, and click **Create**:

- **Language** - sets the base language for the AI action
- **Action type** - sets an action type to serve as a template for the AI action, for example, **Refine action**
- **Model** - sets the AI model used to process the requests resulting from this AI action

1. In the **Global properties** section, set the name and identifier of the AI action.

1. Optionally, provide a description of the AI action.

1. When ready, toggle the status of the AI action to enabled.

1. In the **Settings** area.
For a list of available settings, see [Edit existing AI actions](#edit-existing-ai-actions).

1. Click **Save and close** to apply the changes or **Discard** to discard them and close the window.

7 changes: 7 additions & 0 deletions docs/content_management/block_reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -345,6 +345,13 @@ On the **Properties** tab, set values in the following fields:
- **Content** — Enter a content, for example, text, images, or videos, with the online editor.
For details, see [Editing Rich Text Fields](create_edit_content_items.md#edit-rich-text-fields).

!!! note "Use AI to improve text"

If the [AI Actions](ai_actions.md) LTS update has been installed and configured in your application, the **Improve text** button is visible.
If you have necessary permissions, you can click it to request that the AI Assistant performs specific tasks with your text.

For more information, see [AI Assistant](create_edit_content_items.md#ai-assistant).

## Video block

Integrates a video into the page with standard playback controls.
Expand Down
28 changes: 28 additions & 0 deletions docs/content_management/create_edit_content_items.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,8 @@ To do it, choose one of the available elements:
Each of these elements can have its own settings, such as text formatting.
The option bar also lets you reorder or remove any elements in the Rich Text field.



#### Edit embedded content items

You can edit embedded content items without leaving current window.
Expand Down Expand Up @@ -191,6 +193,32 @@ In table options you can define the first row and/or column as headers, add or d
Your installation can also have custom elements available in the Rich Text field.
Contact your website administrator about the details of using them.

### AI Assistant

If the [AI Actions](ai_actions.md) LTS update has been installed and configured in your application, when writing content, the **Improve text** button is visible in the toolbox.
If you have the required [permissions]([[= developer_doc =]]/permissions/policies/#ai-actions), you can request that the AI Assistant refines your text by using one of the defined actions.
For example, you can highlight a passage and instruct the AI Assistant to extend the passage or adjust its tone to suit your needs.

The **Go to AI actions** button can also visible to users with access to the **Admin UI**.
By clicking it, you can quickly navigate to [AI actions configuration](work_with_ai_actions.md).

![Improve text button and AI actions](ai_assistant.png)

After you select an action, the AI Assistant modal appears, where you can observe the results of AI service's work.
When response is displayed on the screen, depending on whether you are satisfied with the result, you can select one of the options:

- **Replace** - replaces the source text with the output
- **Insert below** - inserts the output text directly after the source passage
- **Try again** - makes another attempt at generating the output

![AI Assistant](ai_assistant_options.png)

!!! note "Lengthy processing of complex requests"

Depending on the length of the source text and the complexity of request, processing may take a while.
You may cancel the processing if it takes too long by clicking **Stop**.


### SeenThis! streaming

!!! note
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
42 changes: 42 additions & 0 deletions docs/getting_started/content_tree/content_tree.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
---
description: Learn about the content tree.
---

# Content tree

Content tree is the place where you can navigate through all the content items in your application.
If you want to browse your application's content items, in the main menu, go to the **Content** -> **Content structure**.
The **Content tree** area is an expandable content menu of your website.

![Content tree in the menu](../img/left_menu_tree.png "Content tree in the menu")

The same tree structure is used in other areas of the Content menu, such as **Media** or **Forms**.

Unique icons for each content type instantly show you what type of content you are selecting. To add custom icons to your content tree, follow the [configuration tutorial in developer documentation]([[= developer_doc =]]/administration/back_office/back_office_elements/custom_icons/#customize-content-type-icons).

For more information about custom configuration, go to [Content tree]([[= developer_doc =]]/administration/back_office/content_tree/) in developer documentation.

Within the content tree, hidden content is greyed out in the tree view.

To simplify the content tree, big lists are collapsed and include a **Show more** icon.
Click it to expand the branches of the tree.
**Collapse all** option, which is available in the context menu, closes all expanded sections.

### Add folders

You can organize the tree by creating additional folders and subfolders of the **Content structure**.
A similar procedure applies to other areas of the system where the tree hierarchy is used.

1\. Go to **Content** -> **Content structure**.

2\. In the upper-right corner click **Create content**.

3\. From the available content types, select **Folder**, and click **Create**.

4\. Enter name for the new folder.

5\. (Optional) Add folder description.

6\. Click **Publish**.

The newly created folder is now visible in the content tree.
15 changes: 3 additions & 12 deletions docs/getting_started/discover_ui.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,21 +67,12 @@ It lets you navigate the content tree, and, for example, create, edit, move, cop

## Content tree

If you want to navigate through your website with a menu, in the main menu, go to the **Content** -> **Content structure**.
The **Content Tree** area is an expandable content menu of your website.
Content tree is the place where you can navigate through all the content items in your application.
It is available in the back office and allows you to browse your content, move content items around, hide them, send them to trash, and perform other actions.

![Content tree in the menu](img/left_menu_tree.png "Content tree in the menu")

Unique icons for each content type instantly show you what type of content you're selecting.
To add custom icons to your content tree, follow [configuration tutorial in Developer Documentation]([[= developer_doc =]]/administration/back_office/back_office_elements/custom_icons/#customize-content-type-icons).

Hidden content is greyed out in the tree view.

To simplify the content tree, big lists are collapsed and include a **Show more** icon.
You can select it to expand the branches of the tree.
**Collapse all** option, which is available in the context menu, closes all expanded sections.

For more information on custom configuration, go to [Content tree]([[= developer_doc =]]/administration/back_office/content_tree/) in Developer Documentation.
For more information, see [Content tree](content_tree.md).

## Content browser

Expand Down
50 changes: 43 additions & 7 deletions docs/image_management/edit_images.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,45 @@ description: Edit images in the Image Editor to flip, crop and select a focal po

# Edit images

When you browse the [Media library](content_model.md#content-and-media), or create or edit a content item that contains an *Image* or *Image asset* field, you can use the Image Editor to perform basic image editing functions.
You can edit the image asset's properties.
You can also edit the image itself with the Image Editor.

The Image Editor enables:
!!! note "Editing tool availability"

The editing tools are available when you work in the back office of your application, for example, to:

- browse the Media root folder of the content tree
- add an image or an image asset to a content item by selecting it from a Content browser or an Image picker
- insert an image into a [Rich Text Field](create_edit_content_items.md#edit-rich-text-fields)

## Edit image asset

You can edit the image asset's properties, for example, to provide a different caption for the image, or change the alternative text for the image.
To edit an existing image asset, in the back office, the main menu, go to **Content** -> **Media** -> **Images**, choose the image asset that you want to edit and click **Edit**.

Here, you can:

- Change the name and caption for the image.

- Edit the image itself in [Image Editor](#edit-image-in-image-editor).

- Change the alternative text, used by screen readers and in SEO.

!!! note "Use AI to generate alternative text"

If the AI Actions LTS update has been installed and configured in your application, the **Use AI** button is visible.
If you have `Action configuration/View` and `Action configuration/Execute` permissions, you can click it to have the alternative text generated automagically.
The **Go to AI actions** button can also visible to users with access to the **Admin UI**.
By clicking it, you can quickly navigate to [AI actions configuration](work_with_ai_actions.md).

![Generate alternative text](img/alt_text_use_ai.png)

- Change the keywords assigned to the image.
Tagging image assets helps keep the media library organized and enables searching images by tags in the DAM.

### Edit image in Image Editor

When you edit the image asset, you can edit the image itself by using the Image Editor that enables:

* image cropping
* image flipping
Expand All @@ -16,12 +52,12 @@ Image Editor is available whenever you see the **Edit** icon on the preview.

![Image Editor icon](image_editor_icon.png)

## Flip
#### Flip

With the Flip feature you mirror the image along a horizontal or vertical axis.
Click either the **Horizontal** or **Vertical** button to flip an image.

## Crop
#### Crop

With the Crop feature you can cut the image down to a desired aspect ratio and dimensions.
You can either choose one of the preset aspect ratio options (square, vertical or horizontal rectangle, or widescreen), or click **Custom**.
Expand All @@ -38,7 +74,7 @@ When you confirm the change, the preview refreshes to display a cropped image.
Confirming the crop operation doesn't mean that it has been saved.
You must click **Save** to apply the changes to the original image.

## Focal point
#### Focal point

If your page contains an image that is larger that the current viewport, for example, when a user accesses the page from a mobile phone, you can select a point on the image that the view should focus on.

Expand All @@ -53,13 +89,13 @@ Click the **Show point** button to see a target on the preview.
Drag the target to a point on the image that you want to focus on.
Click the **X** icon to restore the original position of the target.

## Undo changes
#### Undo changes

Click the left or right arrow button to undo or redo the most recent change.
Click **Reset image** to restore the original appearance of the image.

![Image Editor: Undo](image_editor_undo.png)

## Save changes
#### Save changes

Click **Save** to apply your edits to the original image and exit the Image Editor screen.
Binary file added docs/image_management/img/alt_text_use_ai.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit a38a3b4

Please sign in to comment.