feat: New command `lang:sync` #9023

neznaika0 · 2024-07-05T17:49:58Z

Description
A simple command to copy existing translation files to a new locale.
From the file Language/en/Example.php:

<?php

return [
    'title' => 'Default title',
    'status' => [
        'error'    => 'Error!',
        'done'     => 'Done!',
        'critical' => 'Critical!',
    ],
    'nullableKey' => null,
    'numericKey' => 100500,
    'more' => [
        'nested' => [
            'key' => 'More nested key...',
        ],
    ],
];

After the command php spark lang:sync --locale en --target ru we get Language/ru/Example.php .

<?php

return array (
  'more' => 
  array (
    'nested' => 
    array (
      'key' => 'Example.more.nested.key',
    ),
  ),
  'nullableKey' => 'Example.nullableKey',
  'numericKey' => 'Example.numericKey',
  'status' => 
  array (
    'error' => 'Example.status.error',
    'done' => 'Example.status.done',
    'critical' => 'Example.status.critical',
  ),
  'title' => 'Example.title',
);

applied cs-fix:

<?php

return [
    'more' => [
        'nested' => [
            'key' => 'Example.more.nested.key',
        ],
    ],
    'nullableKey' => 'Example.nullableKey',
    'numericKey'  => 'Example.numericKey',
    'status'      => [
        'error'    => 'Example.status.error',
        'done'     => 'Example.status.done',
        'critical' => 'Example.status.critical',
    ],
    'title' => 'Example.title',
];

And so it is with all other translations

=====

When there are translations in the target folder Language/ru, their keys are saved only if they exist in the Language/en folder:

Language/ru/Example.php:

<?php

return [
    // Deleted keys
    // 'more' => [
    //     'nested' => [
    //         'key' => 'Example.more.nested.key',
    //     ],
    // ],
    'nullableKey' => null,
    'numericKey'  => 100000,
    'status'      => [
        'error'    => 'ru.status.error',
        'done'     => 'ru.status.done',
        'critical' => 'ru.status.critical',
    ],
    'title' => 'ru.title',
    'mistakeKey' => 'ru.mistake_key',
];

Language/ru/Example.php after sync:

<?php

return [
    'more' => [
        'nested' => [
            'key' => 'Example.more.nested.key',
        ],
    ],
    'nullableKey' => null,
    'numericKey'  => 100000,
    'status'      => [
        'error'    => 'ru.status.error',
        'done'     => 'ru.status.done',
        'critical' => 'ru.status.critical',
    ],
    'title' => 'ru.title',
];

Checklist:

Securely signed commits
Component(s) with PHPDoc blocks, only if necessary or adds value
Unit testing, with >80% coverage
User guide updated
Conforms to style guide

neznaika0 · 2024-07-05T17:51:14Z

There are some questions:

Does anyone else need it? 😄
I have indicated the reason for using it as a more precise translation control.
I need help with the documentation. I don't know English
Formatting new arrays requires the use of php-cs-fixer. This was discussed in another PR feat: Language translations finder and update #7896 (comment) . How can we take this out for general use?
phpstan. I don't quite understand what he wants from me. I have specified a common type for the values

neznaika0 · 2024-07-05T18:00:39Z

Userguide:

Changelog:

kenjis · 2024-08-01T02:21:55Z

I don't get the description "When there are translations in the target folder Language/ru, their keys are saved only if they exist in the Language/en folder:". Is that correct?

If app/Language/en/Example.php is:

return [
    // Deleted keys
    // 'more' => [
    //     'nested' => [
    //         'key' => 'Example.more.nested.key',
    //     ],
    // ],
    'nullableKey' => null,
    'numericKey'  => 100000,
    'status'      => [
        'error'    => 'ru.status.error',
        'done'     => 'ru.status.done',
        'critical' => 'ru.status.critical',
    ],
    'title' => 'ru.title',
    'mistakeKey' => 'ru.mistake_key',
];

Then, app/Language/ru/Example.php will be:

return [
    'mistakeKey'  => 'Example.mistakeKey',
    'nullableKey' => 'Example.nullableKey',
    'numericKey'  => 'Example.numericKey',
    'status'      => [
        'error'    => 'Example.status.error',
        'done'     => 'Example.status.done',
        'critical' => 'Example.status.critical',
    ],
    'title' => 'Example.title',
];

That is, deleted keys are gone.

kenjis · 2024-08-01T02:26:50Z

If app/Language/en/Example.php is:

return [
    'title'  => 'Default title',
    'status' => [
        'error'    => 'Error!',
        'done'     => 'Done!',
        'critical' => 'Critical!',
    ],
    'nullableKey' => null,
    'numericKey'  => 100500,
    'more'        => [
        'nested' => [
            'key' => 'More nested key...',
        ],
    ],
];

Then, app/Language/ru/Example.php will be:

return [
    'more' => [
        'nested' => [
            'key' => 'Example.more.nested.key',
        ],
    ],
    'nullableKey' => 'Example.nullableKey',
    'numericKey'  => 'Example.numericKey',
    'status'      => [
        'error'    => 'Example.status.error',
        'done'     => 'Example.status.done',
        'critical' => 'Example.status.critical',
    ],
    'title' => 'Example.title',
];

That is, the key order changed.
This makes comparison of the two files, original and translated, difficult.

user_guide_src/source/outgoing/localization.rst

kenjis · 2024-08-01T02:47:17Z

This command would be useful.
However, I would like the order of the keys to match the original.

neznaika0 · 2024-08-01T05:12:09Z

I don't get the description "When there are translations in the target folder Language/ru, their keys are saved only if they exist in the Language/en folder:". Is that correct?

If app/Language/en/Example.php is:
...

No. Look at the order of the files. We from EN combine RU. There are translations in RU ru.title, so the keys are saved and not overwritten Example.title. Old keys remain.

That is, the key order changed.
This makes comparison of the two files, original and translated, difficult.

Yes. The order has changed. I'm sorting a new array. I think it can be fixed. But the keys will be chaotic when combined.

kenjis · 2024-08-02T07:28:29Z

This PR branch is too old. Can you rebase?

kenjis · 2024-08-02T07:34:49Z

No. Look at the order of the files. We from EN combine RU. There are translations in RU ru.title, so the keys are saved and not overwritten Example.title. Old keys remain.

Okay, I got what you say.

app/Language/en/Example.php:

return [
    'title'  => 'Default title',
    'status' => [
        'error'    => 'Error!',
        'done'     => 'Done!',
        'critical' => 'Critical!',
    ],
    'nullableKey' => null,
    'numericKey'  => 100500,
    'more'        => [
        'nested' => [
            'key' => 'More nested key...',
        ],
    ],
];

app/Language/ru/Example.php:

<?php

return [
    // Deleted keys
    // 'more' => [
    //     'nested' => [
    //         'key' => 'Example.more.nested.key',
    //     ],
    // ],
    'nullableKey' => null,
    'numericKey'  => 100000,
    'status'      => [
        'error'    => 'ru.status.error',
        'done'     => 'ru.status.done',
        'critical' => 'ru.status.critical',
    ],
    'title' => 'ru.title',
    'mistakeKey' => 'ru.mistake_key',
];

$ php spark lang:sync --locale en --target ru
$ composer cs-fix

app/Language/ru/Example.php:

return [
    'more' => [
        'nested' => [
            'key' => 'Example.more.nested.key',
        ],
    ],
    'nullableKey' => null,
    'numericKey'  => 100000,
    'status'      => [
        'error'    => 'ru.status.error',
        'done'     => 'ru.status.done',
        'critical' => 'ru.status.critical',
    ],
    'title' => 'ru.title',
];

kenjis · 2024-08-02T08:04:30Z

Yes. The order has changed. I'm sorting a new array. I think it can be fixed. But the keys will be chaotic when combined.

I think the key order should not be changed.

Can't you rewrite with recursive calls?
This is PoC sample code by ChatGPT:

<?php

function syncTranslationsRecursive($original, $translations) {
    // Initialize an array to store the results
    $syncedTranslations = [];

    // Loop through the original message array
    foreach ($original as $key => $message) {
        if (is_array($message)) {
            // If the message is an array, recursively synchronize it
            $syncedTranslations[$key] = syncTranslationsRecursive(
                $message,
                array_key_exists($key, $translations) ? $translations[$key] : []
            );
        } else {
            // If the message is not an array, check if the translations array contains the same key
            if (array_key_exists($key, $translations)) {
                $syncedTranslations[$key] = $translations[$key];
            } else {
                // If the key doesn't exist in the translations array, set an empty string
                $syncedTranslations[$key] = '';
            }
        }
    }

    return $syncedTranslations;
}

// Original message array
$original = [
    'title' => 'Default title',
    'status' => [
        'error'    => 'Error!',
        'done'     => 'Done!',
        'critical' => 'Critical!',
    ],
    'more' => [
        'nested' => [
            'key' => 'More nested key...',
        ],
    ],
];

// Translated message array (partially translated)
$translations = [
    'title' => 'Título predeterminado',
    'status' => [
        'error' => '¡Error!',
        'done'  => '¡Hecho!',
    ],
];

// Synchronize the translations array with the original message array
$syncedTranslations = syncTranslationsRecursive($original, $translations);

// Print the results
print_r($syncedTranslations);

neznaika0 · 2024-08-02T10:14:35Z

What is the disadvantage of the current array merge?

https://github.com/neznaika0/codeigniter-dev/blob/19f9440bd9deb5131079ff427781f6f14de99584/system/Helpers/Array/ArrayHelper.php#L327-L341

I can just remove the sorting and everything looks like you wanted.
https://github.com/neznaika0/codeigniter-dev/blob/19f9440bd9deb5131079ff427781f6f14de99584/system/Commands/Translation/LocalizationSync.php#L136-L137

But, if the target file has keys, then they are added to the end.

kenjis · 2024-08-02T11:05:08Z

I would like the key order is never changed in any case.
If so, I don't care about the implementation.

neznaika0 · 2024-08-05T05:46:47Z

Results based on the data above:

// translation RU is empty
return [
    'title'  => 'Example.title',
    'status' => [
        'error'    => 'Example.status.error',
        'done'     => 'Example.status.done',
        'critical' => 'Example.status.critical',
    ],
    'nullableKey' => 'Example.nullableKey',
    'numericKey'  => 'Example.numericKey',
    'more'        => [
        'nested' => [
            'key' => 'Example.more.nested.key',
        ],
    ],
];

// translation RU exist
return [
    'nullableKey' => null,
    'numericKey'  => 100000,
    'status'      => [
        'error'    => 'ru.status.error',
        'done'     => 'ru.status.done',
        'critical' => 'ru.status.critical',
    ],
    'title' => 'ru.title',
    'more'  => [
        'nested' => [
            'key' => 'Example.more.nested.key',
        ],
    ],
];

kenjis · 2024-08-05T07:11:58Z

When translation RU exist, the key order will be different from EN.
In any case, the key order should be the same in all lang files.

neznaika0 · 2024-08-08T14:56:15Z

// Before: app/Language/en/Example.php
return [
    'title'  => 'Default title',
    'status' => [
        'error'    => 'Error!',
        'done'     => 'Done!',
        'critical' => 'Critical!',
    ],
    'nullableKey' => null,
    'array'       => [],
    'numericKey'  => 100500,
    'more'        => [
        'nested' => [
            'key' => 'More nested key...',
        ],
    ],
];

// Before: app/Language/ru/Example.php
return [
    // Deleted keys
    // 'more' => [
    //     'nested' => [
    //         'key' => 'Example.more.nested.key',
    //     ],
    // ],
    'nullableKey' => null,
    'status'      => [
        'critical' => 'ru.status.critical',
        'done'     => 'ru.status.done',
        'error'    => 'ru.status.error',
    ],
    'mistakeKey' => 'ru.mistake_key',
    'numericKey'  => 100000,
    'title' => 'ru.title',
];

// After: app/Language/ru/Example.php
// 1. Right sort as in EN
// 2. Delete 'mistakeKey' (in EN language not exist)
// 3. Untranslated keys have a placeholder ''Example.more.nested.key'
// 4. Previously translated keys are saved
return [
    'title'  => 'ru.title',
    'status' => [
        'error'    => 'ru.status.error',
        'done'     => 'ru.status.done',
        'critical' => 'ru.status.critical',
    ],
    'nullableKey' => null,
    'array'       => [
    ],
    'numericKey' => 100000,
    'more'       => [
        'nested' => [
            'key' => 'Example.more.nested.key',
        ],
    ],
];

The array intersection search function ArrayHelper::intersectKeyRecursive is no longer needed.
Combining arrays should now initially not have extra keys (current mistakeKey) from the target array.

datamweb · 2024-08-09T00:29:25Z

I prefer the following result. To make it easier for the developer to translate the file.

// After: app/Language/ru/Example.php
return [
    'title'  => '(To be translated) Default title',
    'status' => [
        'error'    => '(To be translated) Error!',
        'done'     => '(To be translated) Done!',
        'critical' => '(To be translated) Critical!',
    ],
    'nullableKey' => null,
    'array'       => [
    ],
    'numericKey' => 100000,
    'more'       => [
        'nested' => [
            'key' => '(To be translated) More nested key...',
        ],
    ],
];

tests/system/Commands/Translation/LocalizationSyncTest.php

neznaika0 · 2024-08-09T05:46:41Z

@datamweb I think it's wrong. If the current locale is RU and the translations are the same (EN), it is unclear on the page whether the translation was made or not. Because the fallback translation is EN.

Additional text interferes with the preview of the page (too long for some places).

Placeholders clearly show untranslated lines.

It is possible to make it configurable --replace-old

michalsn · 2024-12-21T17:23:43Z

Okay, what is the status of this feature?

It seems like the order of the keys is updated.

Personally, I don't think we need the (To be translated) part as differences in languages are usually quite obvious.

Are there any remaining issues?

neznaika0 · 2024-12-21T18:30:11Z

I'll update the PR in a week. From the corrections: Checking the value is a string. In general, we can abandon PR, since I have a separate repository for lang:find. Synchronization can be moved there.

michalsn · 2024-12-21T18:59:11Z

In general, we can abandon PR, since I have a separate repository for lang:find. Synchronization can be moved there.

So you would rather add this function to the existing lang:find command? Do I understand this correctly?

neznaika0 · 2024-12-22T09:48:03Z

If this command is not needed in CI4, I can add it to the optional package.
The command is separate from lang:find, but in the same group Translation

michalsn

Everything seems to work as required by the previous suggestions, so I think it's fine.

Please just change a bit the user guide part.

michalsn · 2024-12-29T11:11:00Z

user_guide_src/source/outgoing/localization.rst

+After working on your translation for the current language for a long time, in some cases you will need to create files for another language.
+The ``spark lang:find`` command can be used. But it cannot fully detect absolutely all translations. Especially if the parameters are dynamically set as ``lang('App.status.' . $key, ['payload' => 'John'], 'en')``.
+In this case, the best solution is to copy the finished language files and translate them. This way you can save your unique keys that the command did not find.


Suggested change

After working on your translation for the current language for a long time, in some cases you will need to create files for another language.

The ``spark lang:find`` command can be used. But it cannot fully detect absolutely all translations. Especially if the parameters are dynamically set as ``lang('App.status.' . $key, ['payload' => 'John'], 'en')``.

In this case, the best solution is to copy the finished language files and translate them. This way you can save your unique keys that the command did not find.

You may need to create files for another language when you've finished translating for the current language. You can use the spark ``lang:find`` command to help with this. However, it might not detect all translations, particularly those with dynamically set parameters like ``lang('App.status.' . $key, ['payload' => 'John'], 'en')``.

To ensure no translations are missed, it's best to copy the completed language files and translate them manually. This approach preserves any unique keys the command might have overlooked.

ddevsr added the 4.6 label Jul 6, 2024

kenjis added the enhancement PRs that improve existing functionalities label Jul 6, 2024

kenjis reviewed Aug 1, 2024

View reviewed changes

user_guide_src/source/outgoing/localization.rst Outdated Show resolved Hide resolved

datamweb reviewed Aug 1, 2024

View reviewed changes

user_guide_src/source/outgoing/localization.rst Outdated Show resolved Hide resolved

neznaika0 force-pushed the feat-lang-sync branch from cbd2d0f to 19f9440 Compare August 2, 2024 09:49

neznaika0 force-pushed the feat-lang-sync branch from 19f9440 to cf93a62 Compare August 5, 2024 05:37

neznaika0 force-pushed the feat-lang-sync branch from cf93a62 to 6bc2c49 Compare August 8, 2024 14:55

kenjis reviewed Aug 9, 2024

View reviewed changes

tests/system/Commands/Translation/LocalizationSyncTest.php Outdated Show resolved Hide resolved

neznaika0 force-pushed the feat-lang-sync branch from 6bc2c49 to 435d8d6 Compare November 4, 2024 07:09

feat: Add lang:sync command

ebd1828

neznaika0 added 14 commits December 26, 2024 16:57

test: Add tests for lang:sync command

1d175df

feat: Add array helper intersectKeyRecursive

b6d6ce0

docs: Add changelog

07e6321

docs: Add description for command

0ba5bc8

fix: Improve condition

55b97bc

fix: docs title underline too short

f5ea8a8

fix: typo

ba6aa61

fix: Delete debug info

760cbb4

fix: Remove sorting lang keys

c770703

fix: key sorting

e9666f2

Remove array helper intersectKeyRecursive

8da7f75

refactor: Check type translated value

923b5e7

fix: Update tests to be readable

2ee1cb0

fix: Update changelog

6872fa5

neznaika0 force-pushed the feat-lang-sync branch from 435d8d6 to 6872fa5 Compare December 26, 2024 19:14

michalsn requested changes Dec 29, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: New command `lang:sync` #9023

feat: New command `lang:sync` #9023

neznaika0 commented Jul 5, 2024 •

edited by kenjis

Loading

neznaika0 commented Jul 5, 2024 •

edited

Loading

neznaika0 commented Jul 5, 2024

kenjis commented Aug 1, 2024 •

edited

Loading

kenjis commented Aug 1, 2024

kenjis commented Aug 1, 2024

neznaika0 commented Aug 1, 2024

kenjis commented Aug 2, 2024

kenjis commented Aug 2, 2024

kenjis commented Aug 2, 2024

neznaika0 commented Aug 2, 2024

kenjis commented Aug 2, 2024

neznaika0 commented Aug 5, 2024 •

edited

Loading

kenjis commented Aug 5, 2024

neznaika0 commented Aug 8, 2024

datamweb commented Aug 9, 2024 •

edited

Loading

neznaika0 commented Aug 9, 2024 •

edited

Loading

michalsn commented Dec 21, 2024

neznaika0 commented Dec 21, 2024

michalsn commented Dec 21, 2024

neznaika0 commented Dec 22, 2024

michalsn left a comment

michalsn Dec 29, 2024

feat: New command lang:sync #9023

Are you sure you want to change the base?

feat: New command lang:sync #9023

Conversation

neznaika0 commented Jul 5, 2024 • edited by kenjis Loading

neznaika0 commented Jul 5, 2024 • edited Loading

neznaika0 commented Jul 5, 2024

kenjis commented Aug 1, 2024 • edited Loading

kenjis commented Aug 1, 2024

kenjis commented Aug 1, 2024

neznaika0 commented Aug 1, 2024

kenjis commented Aug 2, 2024

kenjis commented Aug 2, 2024

kenjis commented Aug 2, 2024

neznaika0 commented Aug 2, 2024

kenjis commented Aug 2, 2024

neznaika0 commented Aug 5, 2024 • edited Loading

kenjis commented Aug 5, 2024

neznaika0 commented Aug 8, 2024

datamweb commented Aug 9, 2024 • edited Loading

neznaika0 commented Aug 9, 2024 • edited Loading

michalsn commented Dec 21, 2024

neznaika0 commented Dec 21, 2024

michalsn commented Dec 21, 2024

neznaika0 commented Dec 22, 2024

michalsn left a comment

Choose a reason for hiding this comment

michalsn Dec 29, 2024

Choose a reason for hiding this comment

feat: New command `lang:sync` #9023

feat: New command `lang:sync` #9023

neznaika0 commented Jul 5, 2024 •

edited by kenjis

Loading

neznaika0 commented Jul 5, 2024 •

edited

Loading

kenjis commented Aug 1, 2024 •

edited

Loading

neznaika0 commented Aug 5, 2024 •

edited

Loading

datamweb commented Aug 9, 2024 •

edited

Loading

neznaika0 commented Aug 9, 2024 •

edited

Loading