[FEATURE] Integrate media player from slub_web_sachsendigital #1320

fschoelzel · 2024-08-28T15:23:15Z

This PR integrates the video/audio player from slub_web_sachsendigital into a new MediaPlayer plugin.
The PR is based on #796 and #818 from @dvoracek-slub

For testing this within the DFG Viewer, I have prepared a DDEV system at https://github.com/dvoracek-slub/ddev-dfgviewer-dist/tree/mediaplayer. It relies on https://github.com/dvoracek-slub/dfg-viewer/tree/dev-mediaplayer and includes a demo METS video document.

There also is a work-in-progress section in the documentation that describes some features and technical aspects of the player. composer docs:serve can be used for local preview.

Update (2022-07-06). Most notable changes:
* Kitodo in general
  
  * Read production place and person names in `Mods` format class. I use this to test GND linking of some audio-related metadata.
  * Add option to show full TOC.

* Media player
  
  * It is now possible to set and export markers/segments in the media. The list is shown in audio mode (press F2).
  * When the METS links preprocessed waveform data, this can be shown (and be used to edit markers). See the DDEV system for an example.
  * First version of handling recordings that are split across multiple media files. Currently, this only works via page reloads.

github-advanced-security

CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

sebastian-meyer · 2024-09-09T14:31:30Z

You placed a lot of JavaScript files in Resources/Private/JavaScript/. Are you sure those shouldn't be put in Resources/Public/JavaScript/ instead, because they need to be accessible by the client?

Also, please put all unchanged third-party libraries in their own sub-directories separate from your own JS files. There are several reasons for that:

It's easier to update dependencies if you know which files are part of a third-party library and don't contain any custom changes.
We can exclude third-party libraries from all static analysis checks, especially regarding their code styling. (This will most likely greatly reduce the number of Codacy issues currently reported!)

fschoelzel · 2024-09-09T17:21:01Z

You placed a lot of JavaScript files in Resources/Private/JavaScript/. Are you sure those shouldn't be put in Resources/Public/JavaScript/ instead, because they need to be accessible by the client?

This is correct and was done on purpose. All the files in the Resources/Private/JavaScript/ folder are written to a compressed file called DlfMediaPlayer.js during the build process using webpack. This and the associated .map and vendor file are the only files that the client needs when loading the player.
Additional information to code organization: MediaPlayer/Developers.rst

fschoelzel · 2024-09-09T17:48:42Z

Also, please put all unchanged third-party libraries in their own sub-directories separate from your own JS files. There are several reasons for that:

It's easier to update dependencies if you know which files are part of a third-party library and don't contain any custom changes.

We can exclude third-party libraries from all static analysis checks, especially regarding their code styling. (This will most likely greatly reduce the number of Codacy issues currently reported!)

As far as i know this two third-party files were copied from the original repository and rewritten by @dvoracek-slub to meet the ES6 typescript requirements:
Resources/Private/JavaScript/DlfMediaPlayer/3rd-party/EventTarget.js
Resources/Private/JavaScript/DlfMediaPlayer/3rd-party/VideoFrame.js

The only other file I found is:
Resources/Private/JavaScript/SlubMediaPlayer/components/equalizer/filters.js

Which in my opinion should not easily be updated, like the other two files, without deeply testing the whole function of the Player.

Apart from that, all other third-party dependencies are located in the Build/node_modules folder and get compressed into the DlfMediaVendor.js file

fschoelzel · 2024-09-09T17:51:39Z

I agree that the files created after the build process should be stored in the Resources/Private/JavaScript/DlfMediaPlayer folder instead of just the JavaScript folder.

Based upon commit slub/slub_web_sachsendigital#6e68a11

- Adopt build system from `slub_web_sachsendigital` - Use player in plugin - In Table of Contents, link to timecodes - Make element ID configurable

…or retrieving timecode and file IDs

… readability

… entryArray as in Master Branch

… function on MarkerTable

…ata to add timecode and fps

…ions in webpack.config Updated the regular expression in the webpack configuration to ensure that both .css and .less file extensions are correctly anchored at the end of the string. This prevents misleading operator precedence and ensures only the intended files are processed.

… array or null

…oller

…s in MediaPlayerController

- Updated .babelrc to include module-resolver plugin for alias imports. - Modified webpack.config.js to resolve alias paths for cleaner imports. - Update ESLint configuration for improved import resolution and compatibility This change improves the readability and maintainability of import statements by allowing the use of path aliases.

Codacy warning about usage of chr()

fix warning in webpack build: Skipped data-uri embedding of images/xxx.svg because file not found

Replaced PageRenderer with AssetCollector in MediaPlayerController.php for adding CSS and JavaScript assets - Feature TYPO3 v10.3: https://docs.typo3.org/c/typo3/cms-core/main/en-us/Changelog/10.3/Feature-90522-IntroduceAssetCollector.html#changelog-Feature-90522-IntroduceAssetCollector

sebastian-meyer

Some general remarks in addition to the detailed comments below (as discussed in our call):

Please remove the DevServer or make it a dev-only dependency if that's possible.
Please try to merge DlfMediaPlayer, SlubMediaPlayer and lib/ as much as possible, since those are dependencies of each other anyway which won't work as stand-alone implementations.
** At the moment it is really difficult to tell apart what those do, where they are used and how they depend on each other.
** The best way of handling this would be to have just two directories: One containing the third-party Shaka player (and possibly other unchanged third-party libraries) and one containing all the Kitodo-specific code.
Please add license and author information to every non-third-party file.

sebastian-meyer · 2024-09-23T15:36:20Z

.gitignore

 /Build/Test/.env
+composer.lock


Please don't ignore the composer.lock! We want to deliver a known working state of dependencies.

Suggested change

composer.lock

sebastian-meyer · 2024-09-23T16:14:34Z

Classes/Common/MetsDocument.php

@@ -435,6 +435,7 @@ private function getFiles(array &$details, ?SimpleXMLElement $filePointers): voi
            // Check if file has valid @USE attribute.
            if (!empty($fileUse[$fileId])) {
                $details['files'][$fileUse[$fileId]] = $fileId;
+                $details['all_files'][$fileUse[$fileId]][] = $fileId;


I think this is highly confusing. Also, $details['files'] not being an array holding multiple file identifiers may be a bug.

I'd prefer adapting $details['files'] to hold all file identifiers instead of introducing a confusing new variable with just a slightly different use case. What do you think?

sebastian-meyer · 2024-09-23T16:21:18Z

Classes/Common/MetsDocument.php

@@ -1468,6 +1530,7 @@ private function getFileRepresentation(string $id, SimpleXMLElement $physicalNod
            // Check if file has valid @USE attribute.
            if (!empty($fileUse[$fileId])) {
                $this->physicalStructureInfo[$id]['files'][$fileUse[$fileId]] = $fileId;
+                $this->physicalStructureInfo[$id]['all_files'][$fileUse[$fileId]][] = $fileId;


See above. I'd prefer adapting files instead of introducing all_files.

sebastian-meyer · 2024-09-23T16:40:09Z

Classes/ViewHelpers/MediaPlayerConfigViewHelper.php

+
+        $localizationFactory = GeneralUtility::makeInstance(LocalizationFactory::class);
+
+        // TODO: Wouldn't there be a TypoScript utility?


Please have a look at https://docs.typo3.org/other/typo3/view-helper-reference/main/en-us/Global/Translate.html
It's not recommended to implement your own translation function, try to instead use one provided by the TYPO3 framework.

sebastian-meyer · 2024-09-23T16:42:30Z

Configuration/FlexForms/TableOfContents.xml

+                            <label>LLL:EXT:dlf/Resources/Private/Language/locallang_be.xlf:plugins.tableofcontents.flexform.showFull</label>
+                            <config>
+                                <type>check</type>
+                                <default>0</default>


The default here should definitely be 1 in order to keep the current behavior! Please also make sure that this is the default in TableOfContentsController if the option was not set at all (i. e. after upgrading).

sebastian-meyer · 2024-10-21T11:55:57Z

Resources/Private/JavaScript/SlubMediaPlayer/components/equalizer/FrequencyResponse.js

+        // Manual IIR filter
+        // https://www.earlevel.com/main/2016/12/01/evaluating-filter-frequency-response/
+        for (let i = 0; i < this.frequencies_.length; i++) {
+          const f = /** @type {number} */(this.frequencies_[i])


Suggested change

const f = /** @type {number} */(this.frequencies_[i])

const f = /** @type {number} */(this.frequencies_[i]);

sebastian-meyer · 2024-10-21T11:59:15Z

Resources/Private/JavaScript/SlubMediaPlayer/components/equalizer/filtersets/RiaaEq.js

+    // midGain = 10 * Math.log10(1 + (baseBoost / freq) ** 2) - 10 * Math.log10(1 + (baseBoostRolloff / freq) ** 2)
+    // midGain / 10 = Math.log10(1 + (baseBoost / freq) ** 2) - Math.log10(1 + (baseBoostRolloff / freq) ** 2)
+    // midGain / 10 = Math.log10((1 + (baseBoost / freq) ** 2) / (1 + (baseBoostRolloff / freq) ** 2))
+    // 10 ^ (midGain / 10) = (1 + (baseBoost / freq) ** 2) / (1 + (baseBoostRolloff / freq) ** 2)
+
+    // 10 ^ (((10 * Math.log10(1 + baseBoost ** 2) - 10 * Math.log10(1 + baseBoostRolloff ** 2)) / 2) / 10) = (1 + (baseBoost / freq) ** 2) / (1 + (baseBoostRolloff / freq) ** 2)
+    // 10 ^ ((Math.log10((1 + baseBoost ** 2) / (1 + baseBoostRolloff ** 2))) / 2) = (1 + (baseBoost / freq) ** 2) / (1 + (baseBoostRolloff / freq) ** 2)
+    // sqrt((1 + baseBoost ** 2) / (1 + baseBoostRolloff ** 2)) = (1 + (baseBoost / freq) ** 2) / (1 + (baseBoostRolloff / freq) ** 2)
+
+    // For midFreq, ask Wolfram Alpha to solve


sebastian-meyer · 2024-10-21T13:35:43Z

Resources/Private/Less/DlfMediaPlayer.less

@@ -0,0 +1,8 @@
+@import "../../../../slub_digitalcollections/Resources/Private/Less/Helper/Variables.less";


Please don't add dependencies to slub_digitalcollections!

sebastian-meyer · 2024-10-21T13:46:36Z

Tests/Functional/FunctionalTestCase.php

+                'fileGrpVideo' => 'VIDEO,DEFAULT',
+                'fileGrpVideo' => 'VIDEO,DEFAULT,VIDEO_ALT1',


Duplicate use of the array key fileGrpVideo.

sebastian-meyer · 2024-10-21T13:47:57Z

import-mediaplayer.sh

Please remove this file.

github-advanced-security bot found potential problems Aug 28, 2024

View reviewed changes

sebastian-meyer self-requested a review September 9, 2024 14:06

sebastian-meyer assigned fschoelzel Sep 9, 2024

sebastian-meyer added the ⚙ feature A new feature or enhancement. label Sep 9, 2024

sebastian-meyer added this to the Kitodo.Presentation 5.1.0 milestone Sep 9, 2024

fschoelzel force-pushed the dev-mediaplayer branch 3 times, most recently from f8e7a39 to b591463 Compare September 9, 2024 16:09

Kajetan Dvoracek added 18 commits October 9, 2024 15:44

Media Player: Add plugin boilerplate

d17ac6c

Media Player: Add file group, read METS chapters

124045f

fixup "Add file group" [fileGrpVideo]

cf88b1d

Media Player: Import code from slub_web_sachsendigital

c05ff73

Based upon commit slub/slub_web_sachsendigital#6e68a11

Media Player: Integrate imported code

4a8adf2

- Adopt build system from `slub_web_sachsendigital` - Use player in plugin - In Table of Contents, link to timecodes - Make element ID configurable

Media Player: Chapter and page integration

f920ae8

(WIP) Start documenting media player

c1b34ab

(WIP) Add flexforms for player configuration

ab7c410

Media Player: Build JS

47c2845

Configure jsconfig path

0ff87f4

Media Player: Fix and add poster

5484786

Fix: Allow sub-second/float timecode

d1df20f

Read MODS production places to allow linking valueURI

ab7a208

Refactor: Extract share buttons from bookmark modal

a8f8c34

Parse passed time range, not only start

b4bab91

Unify handling of placeholders

65053e7

Fix: Treat HTML as plain text in translation

8588520

Fix: Stop Escape propagation only when actually handled

dcfee9d

fschoelzel added 24 commits October 9, 2024 16:24

fix and refactor MetsDocument.php after rebase

81b4cc5

specify getTimecode function to use fileGrpVideo instead of fileGrp f…

8f9f025

…or retrieving timecode and file IDs

Refactor Mods.php using php-mods-reader to improve code structure and…

01af7d2

… readability

Fix: Refactor Mods.php using php-mods-reader to improve code structure

0048a76

Refactor TableOfContentsController.php to include year information in…

cb3d607

… entryArray as in Master Branch

Refactor MetsDocument to match with Musical Sources Feature

41b4566

Update npm dependencies and Build

16289b3

Media Player: [FEATURE] Select "Remove timestamp" removed in Bookmark…

cd072f7

… function on MarkerTable

Refactor ScreenshotModal.js to fix image filename with extended Metad…

78f1c2f

…ata to add timecode and fps

Rename class name to bookmark-markertable-hidden in the ScreenshotModal

7ee896f

Media Player: Fix and improve getTimecode method to ensure it returns…

2e79d20

… array or null

Media Player: Fix PHPStan Error - PHPDoc comments in MediaPlayerContr…

4beeeb9

…oller

Media Player: Fix PHPStan errors by adding type annotations and check…

476382f

…s in MediaPlayerController

add "video_frame_rate" property in AudioVideoMDTest

b75783d

Media Player: Fixup - Update return type annotation for findFiles method

ab4bdab

Refactor import paths in JavaScript files to path aliases

9729d7e

[MAINTENANCE] Replace chr(31) with pack('C', 31) kitodo#1207

43347a4

Codacy warning about usage of chr()

Refactor: Reduce complexity of getVideoInfo method

7482640

[FIX] Add path for shaka-player/ui to lessOptions in webpack.config

a80aef9

fix warning in webpack build: Skipped data-uri embedding of images/xxx.svg because file not found

Refactor: Update file paths for DlfMediaPlayer JavaScript files

45dc5f5

[STYLE] Update Buttons in BookmarkModal and ScreenshotModal

d51c11c

fschoelzel force-pushed the dev-mediaplayer branch from 78a41f1 to d51c11c Compare October 9, 2024 14:26

sebastian-meyer modified the milestones: Kitodo.Presentation 5.1.0, Kitodo.Presentation 6.0.0 Oct 16, 2024

sebastian-meyer reviewed Oct 21, 2024

View reviewed changes

This was referenced Oct 21, 2024

[FEATURE] Multimedia support #542

Closed

[FEATURE] Integrate media player from slub_web_sachsendigital #818

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FEATURE] Integrate media player from slub_web_sachsendigital #1320

[FEATURE] Integrate media player from slub_web_sachsendigital #1320

fschoelzel commented Aug 28, 2024

github-advanced-security bot left a comment

sebastian-meyer commented Sep 9, 2024 •

edited

Loading

fschoelzel commented Sep 9, 2024 •

edited

Loading

fschoelzel commented Sep 9, 2024

fschoelzel commented Sep 9, 2024

sebastian-meyer left a comment

sebastian-meyer Sep 23, 2024

sebastian-meyer Sep 23, 2024

sebastian-meyer Sep 23, 2024

sebastian-meyer Sep 23, 2024

sebastian-meyer Sep 23, 2024

sebastian-meyer Oct 21, 2024

sebastian-meyer Oct 21, 2024

sebastian-meyer Oct 21, 2024

sebastian-meyer Oct 21, 2024

sebastian-meyer Oct 21, 2024


		$localizationFactory = GeneralUtility::makeInstance(LocalizationFactory::class);

		// TODO: Wouldn't there be a TypoScript utility?

	const f = /** @type {number} */(this.frequencies_[i])
	const f = /** @type {number} */(this.frequencies_[i]);

		@@ -0,0 +1,8 @@
		@import "../../../../slub_digitalcollections/Resources/Private/Less/Helper/Variables.less";

		'fileGrpVideo' => 'VIDEO,DEFAULT',
		'fileGrpVideo' => 'VIDEO,DEFAULT,VIDEO_ALT1',

[FEATURE] Integrate media player from slub_web_sachsendigital #1320

Are you sure you want to change the base?

[FEATURE] Integrate media player from slub_web_sachsendigital #1320

Conversation

fschoelzel commented Aug 28, 2024

github-advanced-security bot left a comment

Choose a reason for hiding this comment

sebastian-meyer commented Sep 9, 2024 • edited Loading

fschoelzel commented Sep 9, 2024 • edited Loading

fschoelzel commented Sep 9, 2024

fschoelzel commented Sep 9, 2024

sebastian-meyer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sebastian-meyer commented Sep 9, 2024 •

edited

Loading

fschoelzel commented Sep 9, 2024 •

edited

Loading