Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Encapsulating PAL/ImageData #15485

Merged
merged 29 commits into from
Aug 14, 2023
Merged

Encapsulating PAL/ImageData #15485

merged 29 commits into from
Aug 14, 2023

Conversation

qiuguohua
Copy link
Contributor

@qiuguohua qiuguohua commented Jun 20, 2023

Re: #
#14646


Continuous Integration

This pull request:

  • needs automatic test cases check.

    Manual trigger with @cocos-robot run test cases afterward.

  • does not change any runtime related code or build configuration

    If any reviewer thinks the CI checks are needed, please uncheck this option, then close and reopen the issue.


Compatibility Check

This pull request:

  • changes public API, and have ensured backward compatibility with deprecated features.
  • affects platform compatibility, e.g. system version, browser version, platform sdk version, platform toolchain, language version, hardware compatibility etc.
  • affects file structure of the build package or build configuration which requires user project upgrade.
  • introduces breaking changes, please list all changes, affected features and the scope of violation.

@github-actions
Copy link

github-actions bot commented Jun 20, 2023

Interface Check Report

! WARNING this pull request has changed these public interfaces:

@@ -744,14 +744,24 @@
     export class SpriteFrame extends Asset {
         /**
          * @en Create a SpriteFrame object by an image asset or an native image asset.
          * @zh 通过 Image 资源或者平台相关 Image 对象创建一个 SpriteFrame 资源。
-         * @param imageSourceOrImageAsset @en ImageAsset or ImageSource, ImageSource could be HTMLCanvasElement, HTMLImageElement, IMemoryImageSource.
-         *                                @zh 图像资源或图像原始图像源,图像原始图像源支持 HTMLCanvasElement HTMLImageElement IMemoryImageSource 三种资源。
+         * @param imageSource @en ImageSource could be HTMLCanvasElement, HTMLImageElement, ImageBitmap.
+         *                    @zh 图像原始图像源支持 HTMLCanvasElement HTMLImageElement ImageBitmap 三种资源。
          * @returns @en SpriteFrame asset. @zh 精灵资源。
+         * @deprecated @en Please use `createWithImage (memoryImageSourceOrImageAsset: ImageAsset | IMemoryImageSource)` instead.
+         *             @zh 请使用`createWithImage (memoryImageSourceOrImageAsset: ImageAsset | IMemoryImageSource)`代替。
          */
-        static createWithImage(imageSourceOrImageAsset: __private._cocos_asset_assets_image_asset__ImageSource | ImageAsset): SpriteFrame;
+        static createWithImage(imageSource: HTMLCanvasElement | HTMLImageElement | ImageBitmap): SpriteFrame;
         /**
+         * @en Create a SpriteFrame object by an image asset or an native image asset.
+         * @zh 通过 Image 资源或者平台相关 Image 对象创建一个 SpriteFrame 资源。
+         * @param memoryImageSourceOrImageAsset @en ImageAsset or IMemoryImageSource.
+         *                                      @zh 图像资产或IMemoryImageSource。
+         * @returns @en SpriteFrame asset. @zh 精灵资源。
+         */
+        static createWithImage(memoryImageSourceOrImageAsset: ImageAsset | __private._pal_image_types__IMemoryImageSource): SpriteFrame;
+        /**
          * @en uv update event.
          * @zh uv 更新事件。
          */
         static EVENT_UV_UPDATED: string;
@@ -9378,8 +9388,16 @@
              * @param regions The region descriptions.
              */
             abstract copyTexImagesToTexture(texImages: Readonly<TexImageSource[]>, texture: Texture, regions: Readonly<BufferTextureCopy[]>): void;
             /**
+             * @en Copy image data to texture.
+             * @zh 拷贝图像资产到纹理。
+             * @param imageData The image data to be copied.
+             * @param texture The texture to copy to.
+             * @param regions The region descriptions.
+             */
+            abstract copyImageDatasToTexture(imageData: Readonly<ImageData[]>, texture: Texture, regions: Readonly<BufferTextureCopy[]>): void;
+            /**
              * @en Whether the device has specific feature.
              * @zh 是否具备特性。
              * @param feature The GFX feature to be queried.
              */
@@ -21320,8 +21338,44 @@
          * 取消注册screen事件回调。
          */
         off(type: __private._pal_screen_adapter_enum_type_screen_event__PalScreenEvent, callback?: (...args: any) => void, target?: any): void;
     }
+    export class ImageData {
+        constructor(source?: __private._pal_image__ImageSource | ArrayBufferView);
+        /**
+         * Destroy resources.
+         */
+        destroy(): void;
+        /**
+         * @en Get raw image data.Return `ImageBitmap` if source is `ImageBitmap`.
+         * @zh 获取原始图像数据。如果source是`ImageBitmap`类型,则返回ImageBitmap。
+         */
+        get data(): __private._pal_image__RawDataType | null;
+        /**
+         * Get and set image data source.
+         * @param value Image data source.
+         * @engineinternal
+         */
+        get source(): __private._pal_image__ImageSource;
+        set source(value: __private._pal_image__ImageSource);
+        /**
+         * Get image width.
+         */
+        get width(): number;
+        /**
+         * Get image height.
+         */
+        get height(): number;
+        /**
+         * Load image via local url or web url.
+         */
+        static loadImage(urlAndBase64: string): Promise<ImageData>;
+        /**
+         * Set image data source.
+         * @param data Image data source.
+         */
+        reset(data?: __private._pal_image__ImageSource | ArrayBufferView): void;
+    }
     /**
      * @en A set of system related variables.
      * @zh 一系列系统相关环境变量。
      */
@@ -28535,13 +28589,26 @@
          */
         get _nativeAsset(): any;
         set _nativeAsset(value: any);
         /**
-         * @en Image data.
-         * @zh 此图像资源的图像数据。
+         * @en Return image data.
+         *     If using a webgl backend:
+         *        - Return source if source is HTMLCanvasElement | HTMLImageElement | ImageBitmap.
+         *        - If source is IMemoryImageSource then return raw image data.
+         *     Other rendering backends that return raw image data.
+         * @zh 返回图像数据。
+         *     如果是webgl的渲染后端:
+         *        - 如果source是HTMLCanvasElement | HTMLImageElement | ImageBitmap则直接返回source.
+         *        - 如果source是IMemoryImageSource则返回原始图像数据.
+         *     其他的渲染后端,返回的都是原始图像数据。
          */
-        get data(): ArrayBufferView | HTMLCanvasElement | HTMLImageElement | ImageBitmap | null;
+        get data(): __private._pal_image_types__ImageSource | __private._pal_image_types__RawDataType | null;
         /**
+         * @en Return image data management object.
+         * @zh 返回图像数据管理对象。
+         */
+        get imageData(): ImageData;
+        /**
          * @en The pixel width of the image.
          * @zh 此图像资源的像素宽度。
          */
         get width(): number;
@@ -28565,15 +28632,34 @@
          * @zh 此图像资源的原始图像源的 URL。当原始图像元不是 HTML 文件时可能为空。
          * @deprecated Please use [[nativeUrl]]
          */
         get url(): string;
-        constructor(nativeAsset?: __private._cocos_asset_assets_image_asset__ImageSource);
         /**
+         * @en Constructing an ImageSource object.
+         * @zh 构造ImageSource对象
+         * @param data @en The image source. @zh 图像数据源。
+         * @deprecated @en please use `constructor(imageSource?: ImageData | IMemoryImageSource)` instead.
+         */
+        constructor(imageSource: HTMLCanvasElement | HTMLImageElement | ImageBitmap);
+        /**
+         * @en Constructing an ImageSource object.
+         * @zh 构造ImageSource对象
+         * @param data @en The image source. @zh 图像数据源。
+         */
+        constructor(imageSource?: ImageData | __private._pal_image_types__IMemoryImageSource);
+        /**
          * @en Reset the source of the image asset.
          * @zh 重置此图像资源使用的原始图像源。
          * @param data @en The new source. @zh 新的图片数据源。
+         * @deprecated @en please use `reset(data: IMemoryImageSource)` instead. @zh 请使用`reset(data: IMemoryImageSource)`替换
          */
-        reset(data: __private._cocos_asset_assets_image_asset__ImageSource): void;
+        reset(data: HTMLCanvasElement | HTMLImageElement | ImageBitmap): void;
+        /**
+         * @en Reset the source of the image asset.
+         * @zh 重置此图像资源使用的原始图像源。
+         * @param data @en The new source. @zh 新的图片数据源。
+         */
+        reset(data: __private._pal_image_types__IMemoryImageSource): void;
         destroy(): boolean;
         initDefault(uuid?: string): void;
         validate(): boolean;
     }
@@ -44156,8 +44242,9 @@
         getBufferBarrier(info: Readonly<gfx.BufferBarrierInfo>): __private._cocos_gfx_base_states_buffer_barrier__BufferBarrier;
         copyBuffersToTexture(buffers: Readonly<ArrayBufferView[]>, texture: gfx.Texture, regions: Readonly<gfx.BufferTextureCopy[]>): void;
         copyTextureToBuffers(texture: Readonly<gfx.Texture>, buffers: ArrayBufferView[], regions: Readonly<gfx.BufferTextureCopy[]>): void;
         copyTexImagesToTexture(texImages: Readonly<TexImageSource[]>, texture: gfx.Texture, regions: Readonly<gfx.BufferTextureCopy[]>): void;
+        copyImageDatasToTexture(imageAssets: Readonly<ImageData[]>, texture: gfx.Texture, regions: Readonly<gfx.BufferTextureCopy[]>): void;
     }
     export class WebGLDevice extends gfx.Device {
         get gl(): WebGLRenderingContext;
         get extensions(): __private._cocos_gfx_webgl_webgl_define__IWebGLExtensions;
@@ -44194,8 +44281,9 @@
         getBufferBarrier(info: Readonly<gfx.BufferBarrierInfo>): __private._cocos_gfx_base_states_buffer_barrier__BufferBarrier;
         copyBuffersToTexture(buffers: Readonly<ArrayBufferView[]>, texture: gfx.Texture, regions: Readonly<gfx.BufferTextureCopy[]>): void;
         copyTextureToBuffers(texture: Readonly<gfx.Texture>, buffers: ArrayBufferView[], regions: Readonly<gfx.BufferTextureCopy[]>): void;
         copyTexImagesToTexture(texImages: Readonly<TexImageSource[]>, texture: gfx.Texture, regions: Readonly<gfx.BufferTextureCopy[]>): void;
+        copyImageDatasToTexture(imageDatas: Readonly<ImageData[]>, texture: gfx.Texture, regions: Readonly<gfx.BufferTextureCopy[]>): void;
     }
     export class WebGL2Device extends gfx.Device {
         get gl(): WebGL2RenderingContext;
         get extensions(): __private._cocos_gfx_webgl2_webgl2_define__IWebGL2Extensions;
@@ -44232,8 +44320,9 @@
         getBufferBarrier(info: Readonly<gfx.BufferBarrierInfo>): __private._cocos_gfx_base_states_buffer_barrier__BufferBarrier;
         copyBuffersToTexture(buffers: Readonly<ArrayBufferView[]>, texture: gfx.Texture, regions: Readonly<gfx.BufferTextureCopy[]>): void;
         copyTextureToBuffers(texture: Readonly<gfx.Texture>, buffers: ArrayBufferView[], regions: Readonly<gfx.BufferTextureCopy[]>): void;
         copyTexImagesToTexture(texImages: Readonly<TexImageSource[]>, texture: gfx.Texture, regions: Readonly<gfx.BufferTextureCopy[]>): void;
+        copyImageDatasToTexture(imageDatas: Readonly<ImageData[]>, texture: gfx.Texture, regions: Readonly<gfx.BufferTextureCopy[]>): void;
     }
     /**
      * @en Intersection2D helper class
      * @zh 辅助类,用于测试形状与形状是否相交
@@ -51059,11 +51148,11 @@
                 blendMode: BlendMode;
                 constructor(index: number, name: string, boneData: BoneData);
             }
             export abstract class Texture {
-                protected _image: HTMLImageElement | ImageBitmap;
-                constructor(image: HTMLImageElement | ImageBitmap);
-                getImage(): HTMLImageElement | ImageBitmap;
+                protected _image: ImageData;
+                constructor(image: ImageData);
+                getImage(): ImageData;
                 abstract setFilters(minFilter: TextureFilter, magFilter: TextureFilter): void;
                 abstract setWraps(uWrap: TextureWrap, vWrap: TextureWrap): void;
                 abstract dispose(): void;
                 static filterFromString(text: string): TextureFilter;
@@ -59250,25 +59339,21 @@
             getLetter(key: any): any;
             getLetterDefinitionForChar(char: any, labelInfo?: any): any;
             clear(): void;
         }
+        /// <reference types="./@types/jsb" />
         /**
          * @en Image source in memory
          * @zh 内存图像源。
          */
-        export interface _cocos_asset_assets_image_asset__IMemoryImageSource {
+        export interface _pal_image_types__IMemoryImageSource {
             _data: ArrayBufferView | null;
             _compressed: boolean;
             width: number;
             height: number;
             format: number;
             mipmapLevelDataSize?: number[];
         }
-        /**
-         * @en The image source, can be HTML canvas, image type or image in memory data
-         * @zh 图像资源的原始图像源。可以来源于 HTML 元素也可以来源于内存。
-         */
-        export type _cocos_asset_assets_image_asset__ImageSource = HTMLCanvasElement | HTMLImageElement | _cocos_asset_assets_image_asset__IMemoryImageSource | ImageBitmap;
         export enum _cocos_2d_assets_sprite_frame__MeshType {
             RECT = 0,
             POLYGON = 1
         }
@@ -61760,8 +61845,10 @@
         export class _cocos_core_event_event_target__Empty {
         }
         export type _cocos_core_event_eventify__Constructor<T> = new (...args: any[]) => T;
         export type _pal_screen_adapter_enum_type_screen_event__PalScreenEvent = "window-resize" | "orientation-change" | "fullscreen-change";
+        export type _pal_image__ImageSource = import('pal/image/types').ImageSource;
+        export type _pal_image__RawDataType = import('pal/image/types').RawDataType;
         export enum _pal_system_info_enum_type_feature__Feature {
             /**
              * @en Feature to support Webp.
              * @zh 是否支持 Webp 特性。
@@ -63619,8 +63706,436 @@
             count: number;
             buffer: Uint8Array;
         }
         /**
+         * @en The image source, can be HTML canvas, image type or image in memory data
+         * @zh 图像资源的原始图像源。可以来源于 HTML 元素也可以来源于内存。
+         */
+        export type _pal_image_types__ImageSource = HTMLCanvasElement | HTMLImageElement | _pal_image_types__IMemoryImageSource | ImageBitmap;
+        export type _types_jsb_jsb__AccelerationXYZ = number;
+        export type _types_jsb_jsb__AccelerationIncludingGravityXYZ = number;
+        export type _types_jsb_jsb__RotationRateAlpha = number;
+        export type _types_jsb_jsb__RotationRateBeta = number;
+        export type _types_jsb_jsb__RotationRateGamma = number;
+        export interface _types_jsb_jsb__MouseEvent {
+            x: number;
+            y: number;
+            xDelta: number | undefined;
+            yDelta: number | undefined;
+            button: number;
+            windowId: number;
+        }
+        export interface _types_jsb_jsb__MouseWheelEvent extends _types_jsb_jsb__MouseEvent {
+            wheelDeltaX: number;
+            wheelDeltaY: number;
+        }
+        export type _types_jsb_jsb__MouseEventCallback = (mouseEvent: _types_jsb_jsb__MouseEvent) => void;
+        export type _types_jsb_jsb__MouseWheelEventCallback = (mouseEvent: _types_jsb_jsb__MouseWheelEvent) => void;
+        export type _types_jsb_jsb__TouchEventCallback = (touchList: TouchList, windowId?: number) => void;
+        export interface _types_jsb_jsb__AxisInfo {
+            code: number;
+            value: number;
+        }
+        export interface _types_jsb_jsb__ButtonInfo {
+            code: number;
+            isPressed: boolean;
+        }
+        export interface _types_jsb_jsb__ControllerInfo {
+            id: number;
+            axisInfoList: _types_jsb_jsb__AxisInfo[];
+            buttonInfoList: _types_jsb_jsb__ButtonInfo[];
+        }
+        export interface _types_jsb_jsb__PoseInfo {
+            code: number;
+            x: number;
+            y: number;
+            z: number;
+            quaternionX: number;
+            quaternionY: number;
+            quaternionZ: number;
+            quaternionW: number;
+        }
+        export interface _types_jsb_jsb__KeyboardEvent {
+            altKey: boolean;
+            ctrlKey: boolean;
+            metaKey: boolean;
+            shiftKey: boolean;
+            repeat: boolean;
+            keyCode: number;
+            windowId: number;
+            code: string;
+        }
+        export type _types_jsb_jsb__KeyboardEventCallback = (keyboardEvent: _types_jsb_jsb__KeyboardEvent) => void;
+        export interface _types_jsb_jsb__WindowEvent {
+            windowId: number;
+            width: number;
+            height: number;
+        }
+        export enum _types_jsb_jsb__AudioFormat {
+            UNKNOWN,
+            SIGNED_8,
+            UNSIGNED_8,
+            SIGNED_16,
+            UNSIGNED_16,
+            SIGNED_32,
+            UNSIGNED_32,
+            FLOAT_32,
+            FLOAT_64
+        }
+        export class _types_jsb_jsb__NativePOD {
+            underlyingData(): ArrayBuffer;
+            _data(): _types_globals__TypedArray;
+            __data: _types_globals__TypedArray;
+        }
+        export class _types_jsb_jsb__Manifest {
+            constructor(manifestUrl: string);
+            constructor(content: string, manifestRoot: string);
+            parseFile(manifestUrl: string): void;
+            parseJSONString(content: string, manifestRoot: string): void;
+            getManifestRoot(): string;
+            getManifestFileUrl(): string;
+            getVersionFileUrl(): string;
+            getSearchPaths(): [
+                string
+            ];
+            getVersion(): string;
+            getPackageUrl(): boolean;
+            setUpdating(isUpdating: boolean): void;
+            isUpdating(): boolean;
+            isVersionLoaded(): boolean;
+            isLoaded(): boolean;
+        }
+        export interface _types_jsb_jsb__ManifestAsset {
+            md5: string;
+            path: string;
+            compressed: boolean;
+            size: number;
+            downloadState: number;
+        }
+        export class _types_jsb_jsb__EventAssetsManager {
+            // EventCode
+            static ERROR_NO_LOCAL_MANIFEST: number;
+            static ERROR_DOWNLOAD_MANIFEST: number;
+            static ERROR_PARSE_MANIFEST: number;
+            static NEW_VERSION_FOUND: number;
+            static ALREADY_UP_TO_DATE: number;
+            static UPDATE_PROGRESSION: number;
+            static ASSET_UPDATED: number;
+            static ERROR_UPDATING: number;
+            static UPDATE_FINISHED: number;
+            static UPDATE_FAILED: number;
+            static ERROR_DECOMPRESS: number;
+            constructor(eventName: string, manager: _types_jsb_jsb__AssetsManager, eventCode: number, assetId?: string, message?: string, curleCode?: number, curlmCode?: number);
+            getAssetsManagerEx(): _types_jsb_jsb__AssetsManager;
+            isResuming(): boolean;
+            getDownloadedFiles(): number;
+            getDownloadedBytes(): number;
+            getTotalFiles(): number;
+            getTotalBytes(): number;
+            getPercent(): number;
+            getPercentByFile(): number;
+            getEventCode(): number;
+            getMessage(): string;
+            getAssetId(): string;
+            getCURLECode(): number;
+            getCURLMCode(): number;
+        }
+        export namespace _types_jsb_jsb__AssetsManager {
+            export enum State {
+                UNINITED,
+                UNCHECKED,
+                PREDOWNLOAD_VERSION,
+                DOWNLOADING_VERSION,
+                VERSION_LOADED,
+                PREDOWNLOAD_MANIFEST,
+                DOWNLOADING_MANIFEST,
+                MANIFEST_LOADED,
+                NEED_UPDATE,
+                READY_TO_UPDATE,
+                UPDATING,
+                UNZIPPING,
+                UP_TO_DATE,
+                FAIL_TO_UPDATE
+            }
+        }
+        export class _types_jsb_jsb__AssetsManager {
+            constructor(manifestUrl: string, storagePath: string, versionCompareHandle?: (versionA: string, versionB: string) => number);
+            static create(manifestUrl: string, storagePath: string): _types_jsb_jsb__AssetsManager;
+            getState(): _types_jsb_jsb__AssetsManager.State;
+            getStoragePath(): string;
+            getMaxConcurrentTask(): number;
+            // setMaxConcurrentTask (max: number): void;  // actually not supported
+            checkUpdate(): void;
+            prepareUpdate(): void;
+            update(): void;
+            isResuming(): boolean;
+            getDownloadedFiles(): number;
+            getDownloadedBytes(): number;
+            getTotalFiles(): number;
+            getTotalBytes(): number;
+            downloadFailedAssets(): void;
+            getLocalManifest(): _types_jsb_jsb__Manifest;
+            loadLocalManifest(manifestUrl: string): boolean;
+            loadLocalManifest(localManifest: _types_jsb_jsb__Manifest, storagePath: string): boolean;
+            getRemoteManifest(): _types_jsb_jsb__Manifest;
+            loadRemoteManifest(remoteManifest: _types_jsb_jsb__Manifest): boolean;
+            /**
+             * Setup your own version compare handler, versionA and B is versions in string.
+             * if the return value greater than 0, versionA is greater than B,
+             * if the return value equals 0, versionA equals to B,
+             * if the return value smaller than 0, versionA is smaller than B.
+             */
+            setVersionCompareHandle(versionCompareHandle?: (versionA: string, versionB: string) => number): void;
+            /**
+             * Setup the verification callback, Return true if the verification passed, otherwise return false
+             */
+            setVerifyCallback(verifyCallback: (path: string, asset: _types_jsb_jsb__ManifestAsset) => boolean): void;
+            setEventCallback(eventCallback: (event: _types_jsb_jsb__EventAssetsManager) => void): void;
+        }
+        export namespace _types_jsb__jsb {
+            let window: any;
+            type AccelerationXYZ = number;
+            type AccelerationIncludingGravityXYZ = number;
+            type RotationRateAlpha = number;
+            type RotationRateBeta = number;
+            type RotationRateGamma = number;
+            type DeviceMotionValue = [
+                _types_jsb_jsb__AccelerationXYZ,
+                _types_jsb_jsb__AccelerationXYZ,
+                _types_jsb_jsb__AccelerationXYZ,
+                _types_jsb_jsb__AccelerationIncludingGravityXYZ,
+                _types_jsb_jsb__AccelerationIncludingGravityXYZ,
+                _types_jsb_jsb__AccelerationIncludingGravityXYZ,
+                _types_jsb_jsb__RotationRateAlpha,
+                _types_jsb_jsb__RotationRateBeta,
+                _types_jsb_jsb__RotationRateGamma
+            ];
+            export interface NativeSafeAreaEdge {
+                /**
+                 * top
+                 */
+                x: number;
+                /**
+                 * left
+                 */
+                y: number;
+                /**
+                 * bottom
+                 */
+                z: number;
+                /**
+                 * right
+                 */
+                w: number;
+            }
+            export interface MouseEvent {
+                x: number;
+                y: number;
+                xDelta: number | undefined;
+                yDelta: number | undefined;
+                button: number;
+                windowId: number;
+            }
+            type MouseEventCallback = (mouseEvent: _types_jsb_jsb__MouseEvent) => void;
+            export interface MouseWheelEvent extends _types_jsb_jsb__MouseEvent {
+                wheelDeltaX: number;
+                wheelDeltaY: number;
+            }
+            type MouseWheelEventCallback = (mouseEvent: _types_jsb_jsb__MouseWheelEvent) => void;
+            export let onMouseDown: _types_jsb_jsb__MouseEventCallback | undefined;
+            export let onMouseMove: _types_jsb_jsb__MouseEventCallback | undefined;
+            export let onMouseUp: _types_jsb_jsb__MouseEventCallback | undefined;
+            export let onMouseWheel: _types_jsb_jsb__MouseWheelEventCallback | undefined;
+            type TouchEventCallback = (touchList: TouchList, windowId?: number) => void;
+            export let onTouchStart: _types_jsb_jsb__TouchEventCallback | undefined;
+            export let onTouchMove: _types_jsb_jsb__TouchEventCallback | undefined;
+            export let onTouchEnd: _types_jsb_jsb__TouchEventCallback | undefined;
+            export let onTouchCancel: _types_jsb_jsb__TouchEventCallback | undefined;
+            export interface ControllerInfo {
+                id: number;
+                axisInfoList: _types_jsb_jsb__AxisInfo[];
+                buttonInfoList: _types_jsb_jsb__ButtonInfo[];
+            }
+            export interface AxisInfo {
+                code: number;
+                value: number;
+            }
+            export interface ButtonInfo {
+                code: number;
+                isPressed: boolean;
+            }
+            export let onControllerInput: (infoList: _types_jsb_jsb__ControllerInfo[]) => void | undefined;
+            export let onHandleInput: (infoList: _types_jsb_jsb__ControllerInfo[]) => void | undefined;
+            export let onControllerChange: (controllerIds: number[]) => void | undefined;
+            export interface PoseInfo {
+                code: number;
+                x: number;
+                y: number;
+                z: number;
+                quaternionX: number;
+                quaternionY: number;
+                quaternionZ: number;
+                quaternionW: number;
+            }
+            export let onHandlePoseInput: (infoList: _types_jsb_jsb__PoseInfo[]) => void | undefined;
+            export let onHMDPoseInput: (infoList: _types_jsb_jsb__PoseInfo[]) => void | undefined;
+            export let onHandheldPoseInput: (infoList: _types_jsb_jsb__PoseInfo[]) => void | undefined;
+            export interface KeyboardEvent {
+                altKey: boolean;
+                ctrlKey: boolean;
+                metaKey: boolean;
+                shiftKey: boolean;
+                repeat: boolean;
+                keyCode: number;
+                windowId: number;
+                code: string;
+            }
+            type KeyboardEventCallback = (keyboardEvent: _types_jsb_jsb__KeyboardEvent) => void;
+            export let onKeyDown: _types_jsb_jsb__KeyboardEventCallback | undefined;
+            export let onKeyUp: _types_jsb_jsb__KeyboardEventCallback | undefined;
+            export interface WindowEvent {
+                windowId: number;
+                width: number;
+                height: number;
+            }
+            export let onResize: (event: _types_jsb_jsb__WindowEvent) => void | undefined;
+            export let onOrientationChanged: (event: {
+                orientation: number;
+            }) => void | undefined;
+            export let onResume: () => void | undefined;
+            export let onPause: () => void | undefined;
+            export let onClose: () => void | undefined;
+            export function openURL(url: string): void;
+            export function garbageCollect(): void;
+            enum AudioFormat {
+                UNKNOWN,
+                SIGNED_8,
+                UNSIGNED_8,
+                SIGNED_16,
+                UNSIGNED_16,
+                SIGNED_32,
+                UNSIGNED_32,
+                FLOAT_32,
+                FLOAT_64
+            }
+            interface PCMHeader {
+                totalFrames: number;
+                sampleRate: number;
+                bytesPerFrame: number;
+                audioFormat: _types_jsb_jsb__AudioFormat;
+                channelCount: number;
+            }
+            class NativePOD {
+                underlyingData(): ArrayBuffer;
+                _data(): _types_globals__TypedArray;
+                __data: _types_globals__TypedArray;
+            }
+            export class Color extends _types_jsb_jsb__NativePOD {
+            }
+            export class Quat extends _types_jsb_jsb__NativePOD {
+            }
+            export class Vec2 extends _types_jsb_jsb__NativePOD {
+            }
+            export class Vec3 extends _types_jsb_jsb__NativePOD {
+            }
+            export class Vec4 extends _types_jsb_jsb__NativePOD {
+            }
+            export class Mat3 extends _types_jsb_jsb__NativePOD {
+            }
+            export class Mat4 extends _types_jsb_jsb__NativePOD {
+            }
+            export interface ManifestAsset {
+                md5: string;
+                path: string;
+                compressed: boolean;
+                size: number;
+                downloadState: number;
+            }
+            export class Manifest {
+                constructor(manifestUrl: string);
+                constructor(content: string, manifestRoot: string);
+                parseFile(manifestUrl: string): void;
+                parseJSONString(content: string, manifestRoot: string): void;
+                getManifestRoot(): string;
+                getManifestFileUrl(): string;
+                getVersionFileUrl(): string;
+                getSearchPaths(): [
+                    string
+                ];
+                getVersion(): string;
+                getPackageUrl(): boolean;
+                setUpdating(isUpdating: boolean): void;
+                isUpdating(): boolean;
+                isVersionLoaded(): boolean;
+                isLoaded(): boolean;
+            }
+            export class EventAssetsManager {
+                // EventCode
+                static ERROR_NO_LOCAL_MANIFEST: number;
+                static ERROR_DOWNLOAD_MANIFEST: number;
+                static ERROR_PARSE_MANIFEST: number;
+                static NEW_VERSION_FOUND: number;
+                static ALREADY_UP_TO_DATE: number;
+                static UPDATE_PROGRESSION: number;
+                static ASSET_UPDATED: number;
+                static ERROR_UPDATING: number;
+                static UPDATE_FINISHED: number;
+                static UPDATE_FAILED: number;
+                static ERROR_DECOMPRESS: number;
+                constructor(eventName: string, manager: _types_jsb_jsb__AssetsManager, eventCode: number, assetId?: string, message?: string, curleCode?: number, curlmCode?: number);
+                getAssetsManagerEx(): _types_jsb_jsb__AssetsManager;
+                isResuming(): boolean;
+                getDownloadedFiles(): number;
+                getDownloadedBytes(): number;
+                getTotalFiles(): number;
+                getTotalBytes(): number;
+                getPercent(): number;
+                getPercentByFile(): number;
+                getEventCode(): number;
+                getMessage(): string;
+                getAssetId(): string;
+                getCURLECode(): number;
+                getCURLMCode(): number;
+            }
+            export class AssetsManager {
+                constructor(manifestUrl: string, storagePath: string, versionCompareHandle?: (versionA: string, versionB: string) => number);
+                static create(manifestUrl: string, storagePath: string): _types_jsb_jsb__AssetsManager;
+                getState(): _types_jsb_jsb__AssetsManager.State;
+                getStoragePath(): string;
+                getMaxConcurrentTask(): number;
+                // setMaxConcurrentTask (max: number): void;  // actually not supported
+                checkUpdate(): void;
+                prepareUpdate(): void;
+                update(): void;
+                isResuming(): boolean;
+                getDownloadedFiles(): number;
+                getDownloadedBytes(): number;
+                getTotalFiles(): number;
+                getTotalBytes(): number;
+                downloadFailedAssets(): void;
+                getLocalManifest(): _types_jsb_jsb__Manifest;
+                loadLocalManifest(manifestUrl: string): boolean;
+                loadLocalManifest(localManifest: _types_jsb_jsb__Manifest, storagePath: string): boolean;
+                getRemoteManifest(): _types_jsb_jsb__Manifest;
+                loadRemoteManifest(remoteManifest: _types_jsb_jsb__Manifest): boolean;
+                /**
+                 * Setup your own version compare handler, versionA and B is versions in string.
+                 * if the return value greater than 0, versionA is greater than B,
+                 * if the return value equals 0, versionA equals to B,
+                 * if the return value smaller than 0, versionA is smaller than B.
+                 */
+                setVersionCompareHandle(versionCompareHandle?: (versionA: string, versionB: string) => number): void;
+                /**
+                 * Setup the verification callback, Return true if the verification passed, otherwise return false
+                 */
+                setVerifyCallback(verifyCallback: (path: string, asset: _types_jsb_jsb__ManifestAsset) => boolean): void;
+                setEventCallback(eventCallback: (event: _types_jsb_jsb__EventAssetsManager) => void): void;
+            }
+            export class JSBNativeDataHolder {
+            }
+        }
+        export type _pal_image_types__RawDataType = ArrayBufferView | _types_jsb__jsb.JSBNativeDataHolder;
+        /**
          * @en The create information for [[Texture2D]].
          * @zh 用来创建贴图的信息。
          */
         export interface _cocos_asset_assets_texture_2d__ITexture2DCreateInfo {
@@ -63707,11 +64222,30 @@
              * - 若图像的尺寸超出了指定层级 Mipmap 的尺寸(不管是长或宽),都将引起错误。
              * @param source @en The source image or image data. @zh 源图像或图像数据。
              * @param level @en Mipmap level to upload the image to. @zh 要上传的 mipmap 层级。
              * @param arrayIndex @en The array index. @zh 要上传的数组索引。
+             * @deprecated since v3.9, please use `uploadData (source: ImageAsset | IMemoryImageSource | ArrayBufferView, level, arrayIndex)` instead.
              */
-            uploadData(source: HTMLCanvasElement | HTMLImageElement | ArrayBufferView | ImageBitmap, level?: number, arrayIndex?: number): void;
+            uploadData(source: HTMLCanvasElement | HTMLImageElement | ImageBitmap, level?: number, arrayIndex?: number): void;
             /**
+             * @en Upload data to the given mipmap level.
+             * The size of the image will affect how the mipmap is updated.
+             * - When the image is an ArrayBuffer, the size of the image must match the mipmap size.
+             * - If the image size matches the mipmap size, the mipmap data will be updated entirely.
+             * - If the image size is smaller than the mipmap size, the mipmap will be updated from top left corner.
+             * - If the image size is larger, an error will be raised
+             * @zh 上传图像数据到指定层级的 Mipmap 中。
+             * 图像的尺寸影响 Mipmap 的更新范围:
+             * - 当图像是 `ArrayBuffer` 时,图像的尺寸必须和 Mipmap 的尺寸一致;否则,
+             * - 若图像的尺寸与 Mipmap 的尺寸相同,上传后整个 Mipmap 的数据将与图像数据一致;
+             * - 若图像的尺寸小于指定层级 Mipmap 的尺寸(不管是长或宽),则从贴图左上角开始,图像尺寸范围内的 Mipmap 会被更新;
+             * - 若图像的尺寸超出了指定层级 Mipmap 的尺寸(不管是长或宽),都将引起错误。
+             * @param source @en The ImageData,IMemoryImageSource or ArrayBufferView. @zh 图像管理对象、内存图像数据或ArrayBufferView。
+             * @param level @en Mipmap level to upload the image to. @zh 要上传的 mipmap 层级。
+             * @param arrayIndex @en The array index. @zh 要上传的数组索引。
+             */
+            uploadData(source: ImageData | _pal_image_types__IMemoryImageSource | ArrayBufferView, level?: number, arrayIndex?: number): void;
+            /**
              * @en Set mipmap level range for this texture.
              * @zh 设置当前贴图的 mipmap 范围。
              * @param baseLevel @en The base mipmap level. @zh 最低 mipmap 等级。
              * @param maxLevel @en The maximum mipmap level. @zh 最高 mipmap 等级。
@@ -64177,9 +64711,9 @@
             deps: string[];
             parsedFromExistAsset?: boolean;
             persistDeps?: string[];
         }
-        export function _cocos_asset_asset_manager_download_dom_image__default(url: string, options: Record<string, any>, onComplete: ((err: Error | null, data?: HTMLImageElement | null) => void)): HTMLImageElement;
+        export function _cocos_asset_asset_manager_download_dom_image__default(url: string, options: Record<string, any>, onComplete: ((err: Error | null, data?: ImageData | null) => void)): void;
         export type _cocos_asset_asset_manager_downloader__DownloadHandler = (url: string, options: Record<string, any>, onComplete: ((err: Error | null, data?: any | null) => void)) => void;
         export type _cocos_asset_asset_manager_download_file__FileProgressCallback = (loaded: number, total: number) => void;
         export function _cocos_asset_asset_manager_download_file__default(url: string, options: Record<string, any>, onProgress: _cocos_asset_asset_manager_download_file__FileProgressCallback | null | undefined, onComplete: ((err: Error | null, data?: any | null) => void)): XMLHttpRequest;
         export function _cocos_asset_asset_manager_download_script__default(url: string, options: Record<string, any>, onComplete: ((err: Error | null, data?: any | null) => void)): HTMLScriptElement | null;

@qiuguohua
Copy link
Contributor Author

@cocos-robot run test cases

@qiuguohua qiuguohua closed this Jun 20, 2023
@qiuguohua qiuguohua reopened this Jun 20, 2023
@qiuguohua
Copy link
Contributor Author

@cocos-robot run test cases

@qiuguohua qiuguohua requested a review from PPpro June 20, 2023 03:17
@github-actions
Copy link

@qiuguohua, Please check the result of run test cases:

Task Details

Platform build boot runned crashScene FailScene
web-mobile PASS PASS FAIL mask-use-image-stencil
windows PASS PASS FAIL label-cacheMode,label-align,ttf-font
ios FAIL PASS PASS
android PASS PASS FAIL label-spacing-x,rich-text-long-string-truncation,mask-type-change,mask-use-image-stencil,sharedMaterial,BuildTimeConstantsTest,rich-text
mac PASS PASS PASS

@github-actions
Copy link

@qiuguohua, Please check the result of run test cases:

Task Details

Platform build boot runned crashScene FailScene
web-mobile PASS PASS FAIL mask-use-image-stencil
windows PASS PASS FAIL label-cacheMode,label-align,ttf-font
ios PASS PASS PASS 2d-rendering-in-3d
android PASS PASS FAIL label-spacing-x,rich-text-long-string-truncation,mask-type-change,mask-use-image-stencil,sharedMaterial,BuildTimeConstantsTest,rich-text
mac PASS PASS PASS

@qiuguohua qiuguohua closed this Jun 20, 2023
@qiuguohua qiuguohua reopened this Jun 20, 2023
@github-actions
Copy link

@qiuguohua, Please check the result of run test cases:

Task Details

Platform build boot runned crashScene FailScene
web-mobile PASS PASS PASS
windows PASS PASS FAIL label-cacheMode,label-align,ttf-font
ios FAIL FAIL
android PASS PASS FAIL label-spacing-x,mask-use-image-stencil
mac PASS PASS PASS

@minggo minggo requested a review from SantyWang June 20, 2023 09:02
@github-actions
Copy link

@qiuguohua, Please check the result of run test cases:

Task Details

Platform build boot runned crashScene FailScene
web-mobile PASS PASS PASS
windows PASS PASS FAIL label-cacheMode,label-align,ttf-font
ios FAIL FAIL
android PASS PASS FAIL label-spacing-x,mask-use-image-stencil
mac FAIL FAIL

@github-actions
Copy link

@qiuguohua, Please check the result of run test cases:

Task Details

Platform build boot runned crashScene FailScene
web-mobile PASS PASS PASS
windows PASS PASS FAIL label-cacheMode,label-align,ttf-font
ios FAIL FAIL
android PASS PASS FAIL label-spacing-x,mask-use-image-stencil
mac PASS PASS PASS

@types/pal/image.d.ts Outdated Show resolved Hide resolved
@types/pal/image.d.ts Outdated Show resolved Hide resolved
@types/pal/image.d.ts Outdated Show resolved Hide resolved
@types/pal/image.d.ts Outdated Show resolved Hide resolved
@@ -0,0 +1,30 @@
declare module 'pal/image' {
Copy link
Contributor

@PPpro PPpro Jun 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's better we add more comment on every 'pal/image' public interface
it's OK to only have en version of comment

the comment sometimes is necessary like what's the difference between data and nativeData
what is format, what number it may returns, what each number represents, is it better to be an enum ?
when to use static method downloadImage, why not to set src directly, or why not use reset instead of setting src

image.d.ts is a design about pal/image, some usage and behavior of interfaces can be fixed in the form of documents to avoid abuse.

@types/pal/image.d.ts Outdated Show resolved Hide resolved
get width(): number;
get height(): number;
set height(value: any);
get format(): number;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this return an enum ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep,need to use enum.

cocos/2d/assets/sprite-frame.ts Outdated Show resolved Hide resolved
@@ -469,7 +469,7 @@ export class SkinnedMeshBatchRenderer extends SkinnedMeshRenderer {
region.texExtent.height = unit.size.y * this.atlasSize;
const { data } = partial.image;
if (!ArrayBuffer.isView(data)) {
texImages.push(data);
texImages.push(data as TexImageSource);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does pal/image module provides a type called TexImageSource ?

It's currently defined in lib.dom.d.ts

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'SourceImage' is partially compatible with 'TexImageSource'. If you don't add 'as' it will report an error.

cocos/asset/assets/simple-texture.ts Outdated Show resolved Hide resolved
cocos/game/splash-screen.ts Outdated Show resolved Hide resolved
cocos/game/splash-screen.ts Outdated Show resolved Hide resolved
@types/pal/image.d.ts Outdated Show resolved Hide resolved
@types/pal/image.d.ts Outdated Show resolved Hide resolved
if (!(value instanceof jsbWindow.HTMLElement) && !isImageBitmap(value)) {
(value as IMemoryImageSource).format = (value as IMemoryImageSource).format || this.format;
}
set (value: any) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better not to use any type

@types/pal/image.d.ts Outdated Show resolved Hide resolved
Comment on lines 607 to 609
if (texture._image instanceof HTMLImageElement) {
texture._image.src = '';
if (JSB) texture._image.destroy();
} else if (sys.hasFeature(sys.Feature.IMAGE_BITMAP) && texture._image instanceof ImageBitmap) {
if (texture._image.close) texture._image.close();
}
texture._image = null;
// TODO:There is a bug here, if you uncomment, it will cause the display result to be incorrect.
// texture.image.destroy();
// texture.image = null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it have any tracking issue ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep,#15324

pal/image/base-image-data.ts Outdated Show resolved Hide resolved
}

set width (value) {
// @ts-ignore
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please don't use ts-ignore

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will report an error because width is a readable property.

pal/image/base-image-data.ts Outdated Show resolved Hide resolved
pal/image/base-image-data.ts Outdated Show resolved Hide resolved
@types/pal/image.d.ts Outdated Show resolved Hide resolved
Comment on lines 65 to 66
image.addEventListener('load', loadCallback);
image.addEventListener('error', errorCallback);
Copy link
Contributor

@PPpro PPpro Jun 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we use onload and onerror ? which is public interface delcared in pal/image module

pal/image/native/image-data.ts Outdated Show resolved Hide resolved
pal/image/native/image-data.ts Outdated Show resolved Hide resolved
2. Modify the dependency of download on HTMLImageElement.
@qiuguohua qiuguohua requested a review from minggo August 10, 2023 06:19
@qiuguohua
Copy link
Contributor Author

@cocos-robot run test cases

@github-actions
Copy link

@qiuguohua, Please check the result of run test cases:

Task Details

Platform build boot runned crashScene FailScene
web-mobile FAIL PASS FAIL compressWithGray,single-compress
web-mobile PASS PASS FAIL compressWithGray,single-compress
windows FAIL PASS FAIL object-group-image,puzzle,shield-node,label-cacheMode,label-align,ttf-font
ios PASS PASS FAIL compressWithGray compressWithGray
android FAIL PASS FAIL object-group-image,puzzle,shield-node,rich-text-long-string-truncation
mac PASS PASS PASS

@github-actions
Copy link

@qiuguohua, Please check the result of run test cases:

Task Details

Platform build boot runned crashScene FailScene
web-mobile PASS PASS PASS
windows PASS PASS FAIL object-group-image,puzzle,shield-node,label-cacheMode,label-align,ttf-font
ios PASS PASS FAIL compressWithGray compressWithGray
android PASS PASS FAIL object-group-image,puzzle,shield-node,rich-text-long-string-truncation
mac PASS PASS PASS

@github-actions
Copy link

@qiuguohua, Please check the result of run test cases:

Task Details

Platform build boot runned crashScene FailScene
web-mobile PASS PASS PASS
windows PASS PASS FAIL object-group-image,puzzle,shield-node,label-cacheMode,label-align,ttf-font
ios PASS PASS FAIL compressWithGray compressWithGray
android PASS PASS FAIL object-group-image,puzzle,shield-node,rich-text-long-string-truncation
mac PASS PASS PASS

@github-actions
Copy link

@qiuguohua, Please check the result of run test cases:

Task Details

Platform build boot runned crashScene FailScene
web-mobile PASS PASS PASS
windows PASS PASS FAIL object-group-image,puzzle,shield-node,label-cacheMode,label-align,ttf-font
ios PASS PASS FAIL compressWithGray compressWithGray
android PASS PASS FAIL object-group-image,puzzle,shield-node,rich-text-long-string-truncation
mac PASS PASS PASS

// TODO: we mark imageAssetProto as type of any, because here we have many dynamic injected property @dumganhar
const imageAssetProto: any = ImageAsset.prototype;

imageAssetProto._ctor = function (nativeAsset?: ImageSource) {
imageAssetProto._ctor = function (nativeAsset?: ImageData | IMemoryImageSource | HTMLCanvasElement | HTMLImageElement | ImageBitmap) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to rename the parameter name nativeAsset.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

@qiuguohua qiuguohua requested a review from minggo August 14, 2023 06:20
@qiuguohua qiuguohua closed this Aug 14, 2023
@qiuguohua qiuguohua reopened this Aug 14, 2023
@sushanhong
Copy link
Contributor

@cocos-robot run test cases

@qiuguohua qiuguohua closed this Aug 14, 2023
@qiuguohua qiuguohua reopened this Aug 14, 2023
@minggo minggo merged commit 0661651 into cocos:develop Aug 14, 2023
67 of 70 checks passed
@github-actions
Copy link

@qiuguohua, Please check the result of run test cases:

Task Details

@github-actions
Copy link

@qiuguohua, Please check the result of run test cases:

Task Details

Platform build boot runned crashScene FailScene
web-mobile PASS PASS FAIL
windows PASS PASS FAIL object-group-image,puzzle,shield-node,label-cacheMode,label-align,ttf-font
ios FAIL FAIL
android PASS PASS FAIL object-group-image,puzzle,shield-node
mac FAIL FAIL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants