Flysystem very slow asset upload to Amazon S3 and Azure Blob Storage with Pimcore 11

[cmann90]

Hello everyone,
we want to migrate from Pimore 10 to 11 and noticed that the asset upload to our Azure Blob Storage is really slow.

Error description

Uploading 5 files with a total of ~50 KB takes ~10-15 seconds. (AWS and Azure)

How to reproduce

1. Install latest Pimcore skeleton with docker images
2. Setup Flysystem for AWS S3 or Azure Blob Storage
3. Asset uploads will take very long

What we tested

Following tests always had the same outcome:

• Our migrated Pimcore 10 to 11 version (Ubuntu VM with Pimcore docker, tested locally and in Azure)
• Pimcore 11 skeleton on docker on local enviroments
  • Platform-version tested: v2023.3, v2024.2, v2024.3
  • Tested with php 8.2 and 8.3 docker containers
  • Flysystem: 3.16, 3.18, 3.29.1
• Pimcore 11 skeleton on docker with AWS S3 instead of Azure Blob Storage on local enviroment

No entry in the logs about the slow upload.

We are trying to find the cause for several days now.
I will post the solution here if we find it.

Thanks in advance already to anyone for their time and input on this.

[fashxp]

Did you also already try a minio/minio on local docker environment? Can be a drop-in replacement for AWS S3, see also https://github.com/pimcore/pimcore/blob/11.x/.github/ci/files/minio-flysystem.yaml.

Maybe this helps to locate the problem.

[cmann90]

Thanks for the hint. We made a comparison with MinIO, AWS and Azure.
The results are a bit strange as Pimcore 11 is faster with MinIO but needs double the time with Azure or AWS:

Image test files (10 jpg files a 295 KB):
test.zip

<?php

namespace App\Command;

use Pimcore\Console\AbstractCommand;
use Pimcore\Model\Asset;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;

class PdnResearchCommand extends AbstractCommand
{
  private $productRelations = array();
  public function configure()
  {
    $this->setName("pdn:research")
      ->setDescription("Research");
  }

  public function execute(InputInterface $input, OutputInterface $output)
  {
    $startTime = microtime(true);

    // Create assets
    $filename = "Media_";
    for ($i = 1; $i <= 10; $i++) {
      $asset = new Asset();
      $asset->setFilename("$filename$i.jpg");
      $imagePath = "./var/test/$filename$i.jpg";
      $imageData = file_get_contents($imagePath);
      $asset->setData($imageData);
      $asset->setParent(Asset::getByPath('/'));
      $asset->save();
    }

    // Delete assets
    for ($i = 1; $i <= 10; $i++) {
      $asset = Asset::getByPath("/$filename$i.jpg");
      if ($asset) {
        $asset->delete();
      }
    }

    $endTime = microtime(true);
    $executionTime = $endTime - $startTime;

    $output->writeln("Time taken to create and delete assets: " . $executionTime . " seconds");

    return self::SUCCESS;
  }
}

[fashxp]

Next step I would do is to repeat the test directly with flysystem bypassing Pimcore...

[cmann90]

Thanks. Sorry for the ping pong here, we are not very experienced with Php, Symfony and Pimcore.

	Pimcore 10	Pimcore 11	Flysystem standalone
Azure	4,01	8,39	1,55
AWS	3,99	9,82	2,47
MinIO	1,53	1,1	0,26

*results are in seconds for an upload and deletion of 10 jpg files with 295 KB each

[fashxp]

Interesting but strange results.
Wondering how it can be, that MinIO is so much faster than AWS whereas Pimcore uses the same code inside Pimcore?

But showing at the MinIO figures for Pimcore 10 and Pimcore 11 it seems to be pretty constant - and afaik in Pimcore there weren't too many changes in that area between 10 and 11.

There some detail debugging and tracing is needed it seems...