From 4acb359a7aa27bf71999641e8c1adac6ae953eca Mon Sep 17 00:00:00 2001 From: Maximilien Date: Tue, 11 Jun 2024 16:40:22 +0200 Subject: [PATCH 01/11] Entry point of Readme purpose --- docs/README.md | 156 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 156 insertions(+) create mode 100644 docs/README.md diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..e2b9fd7 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,156 @@ +# Getting started +## Instalation + +```bash +composer require yokai/batch +``` + +## Vocabulary + +Because when you start with any library +it is important to understand what are the concepts introduced in it. + +This is highly recommended that you read this entire page +before starting to work with this library. + +- [Job](batch/domain/job.md): where you are going to work as a developer. +- [Job Launcher](batch/domain/job-launcher.md): The entry point when you need to execute any job. +- [Job Execution](batch/domain/job-execution.md): The representation of a certain execution of certain job. +- [Job Execution Storage](batch/domain/job-execution-storage.md): The persistence layer of jobs executions. + +## Step by step example +As a developer, from your application, you want to launch a job +```php +launch('import', ['path' => '/path/to/file/to/import']); +``` + +This job will be executed by the `JobLauncher`, and at some point in the call graph, your code will run. This logic have to be implemented in a Job. +```php +getParameter('path'); + // your import logic here + } +}; + +$launcher = new SimpleJobLauncher(...); + +$launcher->launch('import', ['path' => '/path/to/file/to/import']); +``` + +The JobLauncher will have to be provided with all the jobs you create in your application, so it can launch any of it + +:notebook: :notebook: :notebook: + +Note here a container is used to store the jobs, but you can use any PSR-11 container implementation. +For reminder the role of the `JobRegistry` is to register all your job declarations in your container. +For see more refer to [Job Launcher](batch/domain/job-launcher.md) + +:notebook: :notebook: :notebook: + +```php + new class implements JobInterface { + public function execute(JobExecution $jobExecution): void + { + $fileToImport = $jobExecution->getParameter('path'); + // your import logic here + } + }, +]); + +$launcher = new SimpleJobLauncher( + ..., + new JobExecutor( + new JobRegistry($container), + ... + ) +); + +$launcher->launch('import', ['path' => '/path/to/file/to/import']); +``` +But now, what if the job fails, or what if you wish to analyse what the job produced. You need to a able to store JobExecution, so you can fetch it afterwards + +```php + new class implements JobInterface { + public function execute(JobExecution $jobExecution): void + { + $fileToImport = $jobExecution->getParameter('path'); + // your import logic here + } + }, +]); + +$jobExecutionStorage = new FilesystemJobExecutionStorage(new JsonJobExecutionSerializer(), '/dir/where/jobs/are/stored'); +$launcher = new SimpleJobLauncher( + new JobExecutionAccessor( + new JobExecutionFactory(new UniqidJobExecutionIdGenerator(), new NullJobExecutionParametersBuilder()), + $jobExecutionStorage + ), + new JobExecutor( + new JobRegistry($container), + $jobExecutionStorage, + null // or an instance of \Psr\EventDispatcher\EventDispatcherInterface + ) +); + +$importExecution = $launcher->launch('import', ['path' => '/path/to/file/to/import']); +``` + +There you go, you have a fully functional stack to start working with the library. + +[//]: # (Todo::) +[//]: # (## Next steps) + +[//]: # (- [Symfony bridge](symfony.md)) + +[//]: # (- [Doctrine bridge](doctrine.md)) + +[//]: # (- [Flysystem bridge](flysystem.md)) + +[//]: # (- [Box OpenSpout bridge](openspout.md)) From 9ec863b7d9eee650824e1305ab3b998ad8cd2b69 Mon Sep 17 00:00:00 2001 From: Maximilien Date: Tue, 11 Jun 2024 16:49:05 +0200 Subject: [PATCH 02/11] add readme for next steps --- docs/README.md | 11 +++++------ docs/doctrine.md | 0 docs/flysystem.md | 0 docs/openspout.md | 0 docs/symfony.md | 0 5 files changed, 5 insertions(+), 6 deletions(-) create mode 100644 docs/doctrine.md create mode 100644 docs/flysystem.md create mode 100644 docs/openspout.md create mode 100644 docs/symfony.md diff --git a/docs/README.md b/docs/README.md index e2b9fd7..85e948f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -144,13 +144,12 @@ $importExecution = $launcher->launch('import', ['path' => '/path/to/file/to/impo There you go, you have a fully functional stack to start working with the library. -[//]: # (Todo::) -[//]: # (## Next steps) +## Next steps -[//]: # (- [Symfony bridge](symfony.md)) +- [Symfony bridge](symfony.md) -[//]: # (- [Doctrine bridge](doctrine.md)) +- [Doctrine bridge](doctrine.md) -[//]: # (- [Flysystem bridge](flysystem.md)) +- [Flysystem bridge](flysystem.md) -[//]: # (- [Box OpenSpout bridge](openspout.md)) +- [Box OpenSpout bridge](openspout.md) diff --git a/docs/doctrine.md b/docs/doctrine.md new file mode 100644 index 0000000..e69de29 diff --git a/docs/flysystem.md b/docs/flysystem.md new file mode 100644 index 0000000..e69de29 diff --git a/docs/openspout.md b/docs/openspout.md new file mode 100644 index 0000000..e69de29 diff --git a/docs/symfony.md b/docs/symfony.md new file mode 100644 index 0000000..e69de29 From 1a75c8beaed116d3a1d23cf8e965c5c5956372f6 Mon Sep 17 00:00:00 2001 From: Maximilien Date: Wed, 12 Jun 2024 10:11:35 +0200 Subject: [PATCH 03/11] -mv2 of readme ? --- docs/README.md | 165 ++++++++++++++++++++-------------------- docs/getting-started.md | 1 + 2 files changed, 85 insertions(+), 81 deletions(-) create mode 100644 docs/getting-started.md diff --git a/docs/README.md b/docs/README.md index 85e948f..1103912 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,5 +1,5 @@ # Getting started -## Instalation +## Installation ```bash composer require yokai/batch @@ -13,90 +13,85 @@ it is important to understand what are the concepts introduced in it. This is highly recommended that you read this entire page before starting to work with this library. -- [Job](batch/domain/job.md): where you are going to work as a developer. -- [Job Launcher](batch/domain/job-launcher.md): The entry point when you need to execute any job. -- [Job Execution](batch/domain/job-execution.md): The representation of a certain execution of certain job. -- [Job Execution Storage](batch/domain/job-execution-storage.md): The persistence layer of jobs executions. +### Job +A job is the class that is responsible for **what** your code is doing. -## Step by step example -As a developer, from your application, you want to launch a job -```php -launch('import', ['path' => '/path/to/file/to/import']); -``` - -This job will be executed by the `JobLauncher`, and at some point in the call graph, your code will run. This logic have to be implemented in a Job. +For exemple you can have a job for generate a csv: ```php getParameter('path'); - // your import logic here + $file = fopen('path/to/file.csv', 'w'); + + // your export logic here + fputcsv($file, ['column1', 'column2']); + + fclose($file); } -}; - -$launcher = new SimpleJobLauncher(...); - -$launcher->launch('import', ['path' => '/path/to/file/to/import']); +} ``` +#### See More: +For more information about jobs, see [Job](batch/domain/job.md) -The JobLauncher will have to be provided with all the jobs you create in your application, so it can launch any of it +[//]: # (Todo: Maybe we can remove Job.md and put the content here) +### Job Launcher -:notebook: :notebook: :notebook: +The job launcher is responsible for executing/scheduling every jobs. -Note here a container is used to store the jobs, but you can use any PSR-11 container implementation. -For reminder the role of the `JobRegistry` is to register all your job declarations in your container. -For see more refer to [Job Launcher](batch/domain/job-launcher.md) +Yeah, executing OR scheduling. There is multiple implementation of a job launcher across bridges. +Job's execution might be asynchronous, and thus, when you ask the job launcher to "launch" a job, +you have to check the `JobExecution` status that it had returned to know if the job is already executed. -:notebook: :notebook: :notebook: +#### What is needed to use a Job Launcher ? -```php - new class implements JobInterface { +You can start by build a `JobContainer` for store your jobs. +```php +$jobs = new JobContainer([ + 'your.job.name' => new class implements JobInterface { public function execute(JobExecution $jobExecution): void { - $fileToImport = $jobExecution->getParameter('path'); - // your import logic here + // your business logic } }, ]); +``` + +For the storage you can use the `NullJobExecutionStorage`. +```php +$jobExecutionStorage = new NullJobExecutionStorage(); +``` +If you want see more Storage options, see [Job Execution Storage](batch/domain/job-execution-storage.md) +Next, we can build the `JobLauncher` with the `SimpleJobLauncher` implementation and everything else needed to build it. +```php $launcher = new SimpleJobLauncher( - ..., - new JobExecutor( - new JobRegistry($container), - ... - ) + new JobExecutionAccessor(new JobExecutionFactory(new UniqidJobExecutionIdGenerator()), $jobExecutionStorage), + new JobExecutor(new JobRegistry($jobs), $jobExecutionStorage, null), ); - -$launcher->launch('import', ['path' => '/path/to/file/to/import']); ``` -But now, what if the job fails, or what if you wish to analyse what the job produced. You need to a able to store JobExecution, so you can fetch it afterwards + +Finally, you will get a script who seems like this: ```php new class implements JobInterface { +$jobs = new JobContainer([ + 'your.job.name' => new class implements JobInterface { public function execute(JobExecution $jobExecution): void { - $fileToImport = $jobExecution->getParameter('path'); - // your import logic here + // your business logic } }, ]); +$jobExecutionStorage = new NullJobExecutionStorage(); -$jobExecutionStorage = new FilesystemJobExecutionStorage(new JsonJobExecutionSerializer(), '/dir/where/jobs/are/stored'); $launcher = new SimpleJobLauncher( - new JobExecutionAccessor( - new JobExecutionFactory(new UniqidJobExecutionIdGenerator(), new NullJobExecutionParametersBuilder()), - $jobExecutionStorage - ), - new JobExecutor( - new JobRegistry($container), - $jobExecutionStorage, - null // or an instance of \Psr\EventDispatcher\EventDispatcherInterface - ) + new JobExecutionAccessor(new JobExecutionFactory(new UniqidJobExecutionIdGenerator()), $jobExecutionStorage), + new JobExecutor(new JobRegistry($jobs), $jobExecutionStorage, null), ); -$importExecution = $launcher->launch('import', ['path' => '/path/to/file/to/import']); +$execution = $launcher->launch('your.job.name', ['job' => ['configuration']]); ``` +If you want know more you can look one of this: +- [Job Launcher](batch/domain/job-launcher.md) +- [Job Execution](batch/domain/job-execution.md) +- [Job Execution Storage](batch/domain/job-execution-storage.md) -There you go, you have a fully functional stack to start working with the library. +## Step-by-step example ## Next steps - -- [Symfony bridge](symfony.md) - -- [Doctrine bridge](doctrine.md) - +- [Getting Started (Nom de l'exemple)](getting-started.md) +- Framework: + - [Symfony](symfony.md) +- Bridge: + - Doctrine: + - [Dbal](batch-doctrine-dbal/job-execution-storage.md) + - [ORM](batch-doctrine-orm/entity-item-reader.md) + - [Persistence](batch-doctrine-persistence/object-registry.md) + +[//]: # ( Todo: link object-item-writer.md ?) - [Flysystem bridge](flysystem.md) - - [Box OpenSpout bridge](openspout.md) + +[//]: # (Todo: La partie symfony est-elle vraiment utile ? Ou peut être que l'on peut la mettre dans une catégorie a pat entiére ?) +- Symfony: + - [Console](batch-symfony-console/command.md) + - [Messenger](batch-symfony-messenger/job-launcher.md) + - [Serializer](batch-symfony-serializer/job-execution-serializer.md) + - [Validator](batch-symfony-validator/skip-invalid-item-processor.md) + +[//]: # (Idée d'exemple pour le getting started:) +[//]: # (- Un import like Cizeta sans symfony) diff --git a/docs/getting-started.md b/docs/getting-started.md new file mode 100644 index 0000000..593252c --- /dev/null +++ b/docs/getting-started.md @@ -0,0 +1 @@ +//todo From 1be534ee077fe7f06efc66bbfecc967ca1606072 Mon Sep 17 00:00:00 2001 From: Maximilien Date: Wed, 12 Jun 2024 11:16:14 +0200 Subject: [PATCH 04/11] simplify Readme.md --- docs/README.md | 65 +++++++++++++++++++++++--------------------------- 1 file changed, 30 insertions(+), 35 deletions(-) diff --git a/docs/README.md b/docs/README.md index 1103912..6133618 100644 --- a/docs/README.md +++ b/docs/README.md @@ -59,39 +59,12 @@ you have to check the `JobExecution` status that it had returned to know if the For use a Job Launcher you need to have: - **A Container of Jobs:** A container use any PSR-11 [container implementation](https://packagist.org/providers/psr/container-implementation) -- **A Storage for Job Executions:** A storage is a way to store the execution of a job, it can be a database, a file, a cache, etc. -- **A JobExecutor:** The executor is the class that will execute the job. -- **A JobExecutionAccessor:** The accessor is the class that will access the job execution. +- **A Storage for Job Executions:** A storage to store the execution of jobs, it can be a database, a file, a cache, etc. +- **A JobExecutor:** The executor will execute the job. +- **A JobExecutionAccessor:** The accessor will access the job execution. -But don't worry, `Yokai\batch` give you all the tools you need to start. - -You can start by build a `JobContainer` for store your jobs. -```php -$jobs = new JobContainer([ - 'your.job.name' => new class implements JobInterface { - public function execute(JobExecution $jobExecution): void - { - // your business logic - } - }, -]); -``` - -For the storage you can use the `NullJobExecutionStorage`. -```php -$jobExecutionStorage = new NullJobExecutionStorage(); -``` -If you want see more Storage options, see [Job Execution Storage](batch/domain/job-execution-storage.md) - -Next, we can build the `JobLauncher` with the `SimpleJobLauncher` implementation and everything else needed to build it. -```php -$launcher = new SimpleJobLauncher( - new JobExecutionAccessor(new JobExecutionFactory(new UniqidJobExecutionIdGenerator()), $jobExecutionStorage), - new JobExecutor(new JobRegistry($jobs), $jobExecutionStorage, null), -); -``` - -Finally, you will get a script who seems like this: +#### How to use a Job Launcher ? +Look here a simple example to use a Job Launcher with the `SimpleJobLauncher`: ```php launch('your.job.name', ['job' => ['configuration']]); ``` -If you want know more you can look one of this: + +#### See More: - [Job Launcher](batch/domain/job-launcher.md) -- [Job Execution](batch/domain/job-execution.md) + +### JobExecution: + +A [JobExecution](../src/batch/src/JobExecution.php) is the class that holds information about one execution of a job. +#### What kind of information does it hold ? + +- `JobExecution::$jobName` : The Job name (job id) +- `JobExecution::$id` : The execution id +- `JobExecution::$parameters` : Some parameters with which job was executed +- `JobExecution::$status` : A status (pending, running, stopped, completed, abandoned, failed) +- `JobExecution::$startTime` : Start time +- `JobExecution::$endTime` : End time +- `JobExecution::$failures` : A list of failures (usually exceptions) +- `JobExecution::$warnings` : A list of warnings (usually skipped items) +- `JobExecution::$summary` : A summary (can contain any data you wish to store) +- `JobExecution::$logs` : Some logs +- `JobExecution::$childExecutions` : Some child execution + +### JobExecutionStorage + +Whenever a job is launched, whether is starts immediately or not, an execution is stored for it. +The execution are stored to allow you to keep an eye on what is happening. +This persistence is on the responsibility of the job execution storage. - [Job Execution Storage](batch/domain/job-execution-storage.md) -## Step-by-step example ## Next steps - [Getting Started (Nom de l'exemple)](getting-started.md) From 80a1f1aa609f4ea62a645c6f50bf544a02b453b3 Mon Sep 17 00:00:00 2001 From: Maximilien Date: Wed, 12 Jun 2024 11:19:22 +0200 Subject: [PATCH 05/11] remove header of Readme --- docs/README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/README.md b/docs/README.md index 6133618..5bad304 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,4 +1,3 @@ -# Getting started ## Installation ```bash From a8ee53bb6ef9d82ba39bb20bd3d36cf4a7732279 Mon Sep 17 00:00:00 2001 From: Maximilien Date: Wed, 12 Jun 2024 14:43:29 +0200 Subject: [PATCH 06/11] docs/readmeV2 --- README.md | 10 ++- docs/README.md | 110 ++++++++++---------------------- docs/README_old.md | 152 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 190 insertions(+), 82 deletions(-) create mode 100644 docs/README_old.md diff --git a/README.md b/README.md index 13f820c..cb198ae 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,10 @@ This repository contains sources for all packages from `yokai/batch` suite. +## Documentation + +Every package has its own documentation, +you should start with [core repository documentation](https://github.com/yokai-php/batch-src/blob/0.x/docs/README.md). ## Packages @@ -30,12 +34,6 @@ And some special packages : - [`yokai/batch-symfony-pack`](https://github.com/yokai-php/batch-symfony-pack): Minimal pack for Symfony Framework -## Documentation - -Every package has its own documentation, -you should start with [core repository documentation](https://github.com/yokai-php/batch/blob/0.x/README.md). - - ## Contribution Please feel free to open an [issue](https://github.com/yokai-php/batch-src/issues) diff --git a/docs/README.md b/docs/README.md index 5bad304..c7973eb 100644 --- a/docs/README.md +++ b/docs/README.md @@ -4,6 +4,40 @@ composer require yokai/batch ``` +## What is Batch and how does it work? + +```mermaid +sequenceDiagram + participant App + participant SimpleJobLauncher + participant JobExecutionAccessor + participant JobExecutor + participant JobExecutionFactory + participant JobInterface + + App-->>SimpleJobLauncher: Give job name and parameters + activate SimpleJobLauncher + SimpleJobLauncher-->>JobExecutionAccessor: Give job name and parameters + activate JobExecutionAccessor + JobExecutionAccessor-->>JobExecutionStorageInterface: Try to find an existing job for name and id + JobExecutionAccessor-->>JobExecutionFactory: Ask for JobExecution creation if not found in storage + JobExecutionAccessor-->>JobExecutionStorageInterface: Store JobExecution if created + JobExecutionAccessor-->>SimpleJobLauncher: Give back a JobExecution + deactivate JobExecutionAccessor + SimpleJobLauncher-->>JobExecutor: Ask for job execution + activate JobExecutor + JobExecutor-->>JobExecutor: Ensure job can be executed, change job status + JobExecutor-->>JobExecutionStorageInterface: Store JobExecution before job starts + JobExecutor-->>JobInterface: Triggers job execution + JobInterface-->>JobInterface: Where your code lives + JobExecutor-->>JobExecutor: Try/catch all errors and convert to stored failures + JobExecutor-->>JobExecutor: Produces events based on end status of JobExecution + JobExecutor-->>JobExecutionStorageInterface: Store JobExecution after job ends + deactivate JobExecutor + SimpleJobLauncher-->>App: Return the JobExecution, executed + deactivate SimpleJobLauncher +``` + ## Vocabulary Because when you start with any library @@ -20,32 +54,9 @@ as it contains the business logic required for what you wish to achieve. The only requirement is implementing [`JobInterface`](../src/batch/src/Job/JobInterface.php), -For exemple you can have a job for generate a csv: -```php - new class implements JobInterface { - public function execute(JobExecution $jobExecution): void - { - // your business logic - } - }, -]); -$jobExecutionStorage = new NullJobExecutionStorage(); - -$launcher = new SimpleJobLauncher( - new JobExecutionAccessor(new JobExecutionFactory(new UniqidJobExecutionIdGenerator()), $jobExecutionStorage), - new JobExecutor(new JobRegistry($jobs), $jobExecutionStorage, null), -); - -$execution = $launcher->launch('your.job.name', ['job' => ['configuration']]); -``` - -#### See More: -- [Job Launcher](batch/domain/job-launcher.md) - ### JobExecution: A [JobExecution](../src/batch/src/JobExecution.php) is the class that holds information about one execution of a job. -#### What kind of information does it hold ? - -- `JobExecution::$jobName` : The Job name (job id) -- `JobExecution::$id` : The execution id -- `JobExecution::$parameters` : Some parameters with which job was executed -- `JobExecution::$status` : A status (pending, running, stopped, completed, abandoned, failed) -- `JobExecution::$startTime` : Start time -- `JobExecution::$endTime` : End time -- `JobExecution::$failures` : A list of failures (usually exceptions) -- `JobExecution::$warnings` : A list of warnings (usually skipped items) -- `JobExecution::$summary` : A summary (can contain any data you wish to store) -- `JobExecution::$logs` : Some logs -- `JobExecution::$childExecutions` : Some child execution ### JobExecutionStorage diff --git a/docs/README_old.md b/docs/README_old.md new file mode 100644 index 0000000..5bad304 --- /dev/null +++ b/docs/README_old.md @@ -0,0 +1,152 @@ +## Installation + +```bash +composer require yokai/batch +``` + +## Vocabulary + +Because when you start with any library +it is important to understand what are the concepts introduced in it. + +This is highly recommended that you read this entire page +before starting to work with this library. + +### Job +A job is the class that is responsible for **what** your code is doing. + +This is the class you will have to create (or reuse), +as it contains the business logic required for what you wish to achieve. + +The only requirement is implementing [`JobInterface`](../src/batch/src/Job/JobInterface.php), + +For exemple you can have a job for generate a csv: +```php + new class implements JobInterface { + public function execute(JobExecution $jobExecution): void + { + // your business logic + } + }, +]); +$jobExecutionStorage = new NullJobExecutionStorage(); + +$launcher = new SimpleJobLauncher( + new JobExecutionAccessor(new JobExecutionFactory(new UniqidJobExecutionIdGenerator()), $jobExecutionStorage), + new JobExecutor(new JobRegistry($jobs), $jobExecutionStorage, null), +); + +$execution = $launcher->launch('your.job.name', ['job' => ['configuration']]); +``` + +#### See More: +- [Job Launcher](batch/domain/job-launcher.md) + +### JobExecution: + +A [JobExecution](../src/batch/src/JobExecution.php) is the class that holds information about one execution of a job. +#### What kind of information does it hold ? + +- `JobExecution::$jobName` : The Job name (job id) +- `JobExecution::$id` : The execution id +- `JobExecution::$parameters` : Some parameters with which job was executed +- `JobExecution::$status` : A status (pending, running, stopped, completed, abandoned, failed) +- `JobExecution::$startTime` : Start time +- `JobExecution::$endTime` : End time +- `JobExecution::$failures` : A list of failures (usually exceptions) +- `JobExecution::$warnings` : A list of warnings (usually skipped items) +- `JobExecution::$summary` : A summary (can contain any data you wish to store) +- `JobExecution::$logs` : Some logs +- `JobExecution::$childExecutions` : Some child execution + +### JobExecutionStorage + +Whenever a job is launched, whether is starts immediately or not, an execution is stored for it. +The execution are stored to allow you to keep an eye on what is happening. +This persistence is on the responsibility of the job execution storage. +- [Job Execution Storage](batch/domain/job-execution-storage.md) + + +## Next steps +- [Getting Started (Nom de l'exemple)](getting-started.md) +- Framework: + - [Symfony](symfony.md) +- Bridge: + - Doctrine: + - [Dbal](batch-doctrine-dbal/job-execution-storage.md) + - [ORM](batch-doctrine-orm/entity-item-reader.md) + - [Persistence](batch-doctrine-persistence/object-registry.md) + +[//]: # ( Todo: link object-item-writer.md ?) +- [Flysystem bridge](flysystem.md) +- [Box OpenSpout bridge](openspout.md) + +[//]: # (Todo: La partie symfony est-elle vraiment utile ? Ou peut être que l'on peut la mettre dans une catégorie a pat entiére ?) +- Symfony: + - [Console](batch-symfony-console/command.md) + - [Messenger](batch-symfony-messenger/job-launcher.md) + - [Serializer](batch-symfony-serializer/job-execution-serializer.md) + - [Validator](batch-symfony-validator/skip-invalid-item-processor.md) + +[//]: # (Idée d'exemple pour le getting started:) +[//]: # (- Un import like Cizeta sans symfony) From 18e7fae829c8f99c4eb8fb3a9663b15c659b6ebe Mon Sep 17 00:00:00 2001 From: Maximilien Date: Wed, 12 Jun 2024 15:15:50 +0200 Subject: [PATCH 07/11] another version of readme --- docs/README.md | 43 ++++++++++--------------------------------- 1 file changed, 10 insertions(+), 33 deletions(-) diff --git a/docs/README.md b/docs/README.md index c7973eb..25f043a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -4,39 +4,16 @@ composer require yokai/batch ``` -## What is Batch and how does it work? - -```mermaid -sequenceDiagram - participant App - participant SimpleJobLauncher - participant JobExecutionAccessor - participant JobExecutor - participant JobExecutionFactory - participant JobInterface - - App-->>SimpleJobLauncher: Give job name and parameters - activate SimpleJobLauncher - SimpleJobLauncher-->>JobExecutionAccessor: Give job name and parameters - activate JobExecutionAccessor - JobExecutionAccessor-->>JobExecutionStorageInterface: Try to find an existing job for name and id - JobExecutionAccessor-->>JobExecutionFactory: Ask for JobExecution creation if not found in storage - JobExecutionAccessor-->>JobExecutionStorageInterface: Store JobExecution if created - JobExecutionAccessor-->>SimpleJobLauncher: Give back a JobExecution - deactivate JobExecutionAccessor - SimpleJobLauncher-->>JobExecutor: Ask for job execution - activate JobExecutor - JobExecutor-->>JobExecutor: Ensure job can be executed, change job status - JobExecutor-->>JobExecutionStorageInterface: Store JobExecution before job starts - JobExecutor-->>JobInterface: Triggers job execution - JobInterface-->>JobInterface: Where your code lives - JobExecutor-->>JobExecutor: Try/catch all errors and convert to stored failures - JobExecutor-->>JobExecutor: Produces events based on end status of JobExecution - JobExecutor-->>JobExecutionStorageInterface: Store JobExecution after job ends - deactivate JobExecutor - SimpleJobLauncher-->>App: Return the JobExecution, executed - deactivate SimpleJobLauncher -``` +## What is Batch ? +The Batch library solves all your massive data processing problems. + +Batch can also be used as an ETL. + +Batch can also work asynchronously. + +## How it works ? +Batch is a library that allows you to declare and execute jobs. +And having control over each step of the process to be able to extend the technical logic to meet business needs. ## Vocabulary From 06768eb5de424b763d3fb170229260106dedfab5 Mon Sep 17 00:00:00 2001 From: Maximilien Date: Wed, 12 Jun 2024 15:57:31 +0200 Subject: [PATCH 08/11] fix docs/readme --- docs/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/README.md b/docs/README.md index 25f043a..0e6504d 100644 --- a/docs/README.md +++ b/docs/README.md @@ -11,6 +11,8 @@ Batch can also be used as an ETL. Batch can also work asynchronously. +Batch help you to make reporting during process. + ## How it works ? Batch is a library that allows you to declare and execute jobs. And having control over each step of the process to be able to extend the technical logic to meet business needs. From 43f17ec6e6b28eff6c827ae0b9fafff21ac89604 Mon Sep 17 00:00:00 2001 From: Maximilien Date: Wed, 12 Jun 2024 17:56:58 +0200 Subject: [PATCH 09/11] refacto readme.md --- docs/doctrine.md | 0 docs/flysystem.md | 0 docs/getting-started.md | 1 - .../project-overview.md} | 79 ++++--------------- docs/openspout.md | 0 docs/symfony.md | 0 6 files changed, 14 insertions(+), 66 deletions(-) delete mode 100644 docs/doctrine.md delete mode 100644 docs/flysystem.md delete mode 100644 docs/getting-started.md rename docs/{README_old.md => getting-started/project-overview.md} (54%) delete mode 100644 docs/openspout.md delete mode 100644 docs/symfony.md diff --git a/docs/doctrine.md b/docs/doctrine.md deleted file mode 100644 index e69de29..0000000 diff --git a/docs/flysystem.md b/docs/flysystem.md deleted file mode 100644 index e69de29..0000000 diff --git a/docs/getting-started.md b/docs/getting-started.md deleted file mode 100644 index 593252c..0000000 --- a/docs/getting-started.md +++ /dev/null @@ -1 +0,0 @@ -//todo diff --git a/docs/README_old.md b/docs/getting-started/project-overview.md similarity index 54% rename from docs/README_old.md rename to docs/getting-started/project-overview.md index 5bad304..34fe43b 100644 --- a/docs/README_old.md +++ b/docs/getting-started/project-overview.md @@ -1,26 +1,18 @@ -## Installation +# Getting Started -```bash -composer require yokai/batch -``` - -## Vocabulary +## Project Overview -Because when you start with any library -it is important to understand what are the concepts introduced in it. +For a step-by-step presentation of the batch library, we're going to focus on a simple requirement. +Update products and stocks via an Excel import. -This is highly recommended that you read this entire page -before starting to work with this library. -### Job +### What is a job technically ? A job is the class that is responsible for **what** your code is doing. This is the class you will have to create (or reuse), as it contains the business logic required for what you wish to achieve. -The only requirement is implementing [`JobInterface`](../src/batch/src/Job/JobInterface.php), - -For exemple you can have a job for generate a csv: +The only requirement is implementing [`JobInterface`](../../src/batch/src/Job/JobInterface.php), ```php launch('your.job.name', ['job' => ['configuration']]); ``` -#### See More: -- [Job Launcher](batch/domain/job-launcher.md) - -### JobExecution: +#### More information you need know: +##### JobExecution: -A [JobExecution](../src/batch/src/JobExecution.php) is the class that holds information about one execution of a job. +A [JobExecution](../../src/batch/src/JobExecution.php) is the class that holds information about one execution of a job. #### What kind of information does it hold ? - `JobExecution::$jobName` : The Job name (job id) @@ -119,34 +93,9 @@ A [JobExecution](../src/batch/src/JobExecution.php) is the class that holds info - `JobExecution::$logs` : Some logs - `JobExecution::$childExecutions` : Some child execution -### JobExecutionStorage +#### JobExecutionStorage Whenever a job is launched, whether is starts immediately or not, an execution is stored for it. The execution are stored to allow you to keep an eye on what is happening. This persistence is on the responsibility of the job execution storage. -- [Job Execution Storage](batch/domain/job-execution-storage.md) - - -## Next steps -- [Getting Started (Nom de l'exemple)](getting-started.md) -- Framework: - - [Symfony](symfony.md) -- Bridge: - - Doctrine: - - [Dbal](batch-doctrine-dbal/job-execution-storage.md) - - [ORM](batch-doctrine-orm/entity-item-reader.md) - - [Persistence](batch-doctrine-persistence/object-registry.md) - -[//]: # ( Todo: link object-item-writer.md ?) -- [Flysystem bridge](flysystem.md) -- [Box OpenSpout bridge](openspout.md) - -[//]: # (Todo: La partie symfony est-elle vraiment utile ? Ou peut être que l'on peut la mettre dans une catégorie a pat entiére ?) -- Symfony: - - [Console](batch-symfony-console/command.md) - - [Messenger](batch-symfony-messenger/job-launcher.md) - - [Serializer](batch-symfony-serializer/job-execution-serializer.md) - - [Validator](batch-symfony-validator/skip-invalid-item-processor.md) - -[//]: # (Idée d'exemple pour le getting started:) -[//]: # (- Un import like Cizeta sans symfony) +- [Job Execution Storage](../batch/domain/job-execution-storage.md) diff --git a/docs/openspout.md b/docs/openspout.md deleted file mode 100644 index e69de29..0000000 diff --git a/docs/symfony.md b/docs/symfony.md deleted file mode 100644 index e69de29..0000000 From 578ebe554b586b89e7c007574a690b9efe0d9976 Mon Sep 17 00:00:00 2001 From: Maximilien Date: Wed, 12 Jun 2024 17:57:35 +0200 Subject: [PATCH 10/11] reafcto readme.md --- docs/getting-started/project-overview.md | 101 ----------------------- 1 file changed, 101 deletions(-) delete mode 100644 docs/getting-started/project-overview.md diff --git a/docs/getting-started/project-overview.md b/docs/getting-started/project-overview.md deleted file mode 100644 index 34fe43b..0000000 --- a/docs/getting-started/project-overview.md +++ /dev/null @@ -1,101 +0,0 @@ -# Getting Started - -## Project Overview - -For a step-by-step presentation of the batch library, we're going to focus on a simple requirement. -Update products and stocks via an Excel import. - - -### What is a job technically ? -A job is the class that is responsible for **what** your code is doing. - -This is the class you will have to create (or reuse), -as it contains the business logic required for what you wish to achieve. - -The only requirement is implementing [`JobInterface`](../../src/batch/src/Job/JobInterface.php), -```php - new class implements JobInterface { - public function execute(JobExecution $jobExecution): void - { - // your business logic - } - }, -]); -$jobExecutionStorage = new NullJobExecutionStorage(); - -$launcher = new SimpleJobLauncher( - new JobExecutionAccessor(new JobExecutionFactory(new UniqidJobExecutionIdGenerator()), $jobExecutionStorage), - new JobExecutor(new JobRegistry($jobs), $jobExecutionStorage, null), -); - -$execution = $launcher->launch('your.job.name', ['job' => ['configuration']]); -``` - -#### More information you need know: -##### JobExecution: - -A [JobExecution](../../src/batch/src/JobExecution.php) is the class that holds information about one execution of a job. -#### What kind of information does it hold ? - -- `JobExecution::$jobName` : The Job name (job id) -- `JobExecution::$id` : The execution id -- `JobExecution::$parameters` : Some parameters with which job was executed -- `JobExecution::$status` : A status (pending, running, stopped, completed, abandoned, failed) -- `JobExecution::$startTime` : Start time -- `JobExecution::$endTime` : End time -- `JobExecution::$failures` : A list of failures (usually exceptions) -- `JobExecution::$warnings` : A list of warnings (usually skipped items) -- `JobExecution::$summary` : A summary (can contain any data you wish to store) -- `JobExecution::$logs` : Some logs -- `JobExecution::$childExecutions` : Some child execution - -#### JobExecutionStorage - -Whenever a job is launched, whether is starts immediately or not, an execution is stored for it. -The execution are stored to allow you to keep an eye on what is happening. -This persistence is on the responsibility of the job execution storage. -- [Job Execution Storage](../batch/domain/job-execution-storage.md) From f9412e26245eb34cbec56bab5ccf3f8b570b4000 Mon Sep 17 00:00:00 2001 From: Maximilien Date: Thu, 13 Jun 2024 10:15:10 +0200 Subject: [PATCH 11/11] Update ReadmeDocumentation --- README.md | 3 +-- docs/README.md | 25 ------------------------- 2 files changed, 1 insertion(+), 27 deletions(-) diff --git a/README.md b/README.md index cb198ae..05c1df4 100644 --- a/README.md +++ b/README.md @@ -8,8 +8,7 @@ This repository contains sources for all packages from `yokai/batch` suite. ## Documentation -Every package has its own documentation, -you should start with [core repository documentation](https://github.com/yokai-php/batch-src/blob/0.x/docs/README.md). +You should start with [centralized documentation](https://github.com/yokai-php/batch-src/blob/0.x/docs/README.md). ## Packages diff --git a/docs/README.md b/docs/README.md index 0e6504d..bf030ad 100644 --- a/docs/README.md +++ b/docs/README.md @@ -62,28 +62,3 @@ Whenever a job is launched, whether is starts immediately or not, an execution i The execution are stored to allow you to keep an eye on what is happening. This persistence is on the responsibility of the job execution storage. - [Job Execution Storage](batch/domain/job-execution-storage.md) - - -## Next steps -- [Getting Started (Nom de l'exemple)](getting-started.md) -- Framework: - - [Symfony](symfony.md) -- Bridge: - - Doctrine: - - [Dbal](batch-doctrine-dbal/job-execution-storage.md) - - [ORM](batch-doctrine-orm/entity-item-reader.md) - - [Persistence](batch-doctrine-persistence/object-registry.md) - -[//]: # ( Todo: link object-item-writer.md ?) -- [Flysystem bridge](flysystem.md) -- [Box OpenSpout bridge](openspout.md) - -[//]: # (Todo: La partie symfony est-elle vraiment utile ? Ou peut être que l'on peut la mettre dans une catégorie a pat entiére ?) -- Symfony: - - [Console](batch-symfony-console/command.md) - - [Messenger](batch-symfony-messenger/job-launcher.md) - - [Serializer](batch-symfony-serializer/job-execution-serializer.md) - - [Validator](batch-symfony-validator/skip-invalid-item-processor.md) - -[//]: # (Idée d'exemple pour le getting started:) -[//]: # (- Un import like Cizeta sans symfony)