Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In Liblouis 3.30 release 18 Liblouisutdml Math Voluve tests are failing #106

Closed
hammera opened this issue Jun 4, 2024 · 18 comments · Fixed by #107
Closed

In Liblouis 3.30 release 18 Liblouisutdml Math Voluve tests are failing #106

hammera opened this issue Jun 4, 2024 · 18 comments · Fixed by #107

Comments

@hammera
Copy link
Contributor

hammera commented Jun 4, 2024

Hi Boys,

Before I dropping a general version number update related PR to the Liblouis 3.30 version number the Liblouisutdml repository affected files (Dockerfile.win32, Dockerfile.win64, .github/workflow/main.iml, .github.workflow/sanitizer.iml), I compiled my Debian system a Liblouis 3.30 deb package and install the updated package version.
Make command ran successfull the Liblouisutdml source tree, but when I ran the test directory the make check command, I see following test-suite.log:

================================================
   liblouisutdml 2.13.0: tests/test-suite.log
================================================

# TOTAL: 386
# PASS:  187
# SKIP:  0
# XFAIL: 181
# FAIL:  18
# XPASS: 0
# ERROR: 0

Me suspicious happens the failure because happened a change the virtual dots feature related, this situations always the math_voluwe tests produced failures.
Bingo. :-):-)
Following the failing tests list:

  • test_mathml_woluwe/test_051
  • test_mathml_woluwe/test_088
  • test_mathml_woluwe/test_090
  • test_mathml_woluwe/test_092
  • test_mathml_woluwe/test_094
  • test_mathml_woluwe/test_095
  • test_mathml_woluwe/test_097
  • test_mathml_woluwe/test_098
  • test_mathml_woluwe/test_101
  • test_mathml_woluwe/test_102
  • test_mathml_woluwe/test_103
  • test_mathml_woluwe/test_104
  • test_mathml_woluwe/test_105
  • test_mathml_woluwe/test_108
  • test_mathml_woluwe/test_109
  • test_mathml_woluwe/test_110
  • test_mathml_woluwe/test_111
  • test_mathml_woluwe/test_119

Looks the verbose tests (only the failures part):

FAIL: test_mathml_woluwe/test_051
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_051.test/expected.txt	2022-12-12 12:31:51.736096753 +0100
+++ /tmp/foosWbq3i/output.txt	2024-06-04 13:15:03.942981180 +0200
@@ -1 +1 @@
-@N"*"(#J")"7@N*0
+@N"*"(#J")"7@NA_R0
FAIL test_mathml_woluwe/test_051.test (exit status: 1)

FAIL: test_mathml_woluwe/test_088
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_088.test/expected.txt	2022-12-12 12:31:51.736096753 +0100
+++ /tmp/foomNK5c2/output.txt	2024-06-04 13:15:04.746985928 +0200
@@ -1 +1 @@
-X*N:
+XA_RN:
FAIL test_mathml_woluwe/test_088.test (exit status: 1)

FAIL: test_mathml_woluwe/test_090
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_090.test/expected.txt	2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooH5928L/output.txt	2024-06-04 13:15:04.794986212 +0200
@@ -1 +1 @@
-#*N:X
+A_LN:X
FAIL test_mathml_woluwe/test_090.test (exit status: 1)

FAIL: test_mathml_woluwe/test_092
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_092.test/expected.txt	2022-12-12 12:31:51.736096753 +0100
+++ /tmp/foo4pXn5P/output.txt	2024-06-04 13:15:04.834986449 +0200
@@ -1 +1 @@
-X_*N:
+XA_CN:
FAIL test_mathml_woluwe/test_092.test (exit status: 1)

FAIL: test_mathml_woluwe/test_094
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_094.test/expected.txt	2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooDLv7PI/output.txt	2024-06-04 13:15:04.882986732 +0200
@@ -1 +1 @@
-#*N:X*M:
+A_LN:XA_RM:
FAIL test_mathml_woluwe/test_094.test (exit status: 1)

FAIL: test_mathml_woluwe/test_095
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_095.test/expected.txt	2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooARDilN/output.txt	2024-06-04 13:15:04.902986850 +0200
@@ -1 +1 @@
-#*N:X*M:
+A_LN:XA_RM:
FAIL test_mathml_woluwe/test_095.test (exit status: 1)

FAIL: test_mathml_woluwe/test_097
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_097.test/expected.txt	2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooO4Zk1d/output.txt	2024-06-04 13:15:04.946987112 +0200
@@ -1 +1 @@
-#*N:X_/M:
+A_LN:X_/M:
FAIL test_mathml_woluwe/test_097.test (exit status: 1)

FAIL: test_mathml_woluwe/test_098
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_098.test/expected.txt	2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooCloX1f/output.txt	2024-06-04 13:15:04.970987254 +0200
@@ -1 +1 @@
-X/N:*M:/P:
+X/N:A_RM:/P:
FAIL test_mathml_woluwe/test_098.test (exit status: 1)

FAIL: test_mathml_woluwe/test_101
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_101.test/expected.txt	2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooxXvvIW/output.txt	2024-06-04 13:15:05.038987654 +0200
@@ -1 +1 @@
-X*1,1 X*2
+XA_R1,1 XA_R2
FAIL test_mathml_woluwe/test_101.test (exit status: 1)

FAIL: test_mathml_woluwe/test_102
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_102.test/expected.txt	2022-12-12 12:31:51.736096753 +0100
+++ /tmp/foonPFWac/output.txt	2024-06-04 13:15:05.062987796 +0200
@@ -1 +1 @@
-X*2/3
+XA_R2/3
FAIL test_mathml_woluwe/test_102.test (exit status: 1)

FAIL: test_mathml_woluwe/test_103
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_103.test/expected.txt	2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooV5G522/output.txt	2024-06-04 13:15:05.082987913 +0200
@@ -1 +1 @@
-X*2/3
+XA_R2/3
FAIL test_mathml_woluwe/test_103.test (exit status: 1)

FAIL: test_mathml_woluwe/test_104
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_104.test/expected.txt	2022-12-12 12:31:51.736096753 +0100
+++ /tmp/foo7DTMm8/output.txt	2024-06-04 13:15:05.106988055 +0200
@@ -1 +1 @@
-X*P:Y/Q:
+XA_RP:Y/Q:
FAIL test_mathml_woluwe/test_104.test (exit status: 1)

FAIL: test_mathml_woluwe/test_105
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_105.test/expected.txt	2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooTNnpYQ/output.txt	2024-06-04 13:15:05.130988197 +0200
@@ -1 +1 @@
-A*N"-#A:X/N"-#A:
+AA_RN"-#A:X/N"-#A:
FAIL test_mathml_woluwe/test_105.test (exit status: 1)

FAIL: test_mathml_woluwe/test_108
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_108.test/expected.txt	2022-12-12 12:31:51.736096753 +0100
+++ /tmp/foo8oHED1/output.txt	2024-06-04 13:15:05.198988598 +0200
@@ -1 +1 @@
-A*X"6Y:/Z:
+AA_RX"6Y:/Z:
FAIL test_mathml_woluwe/test_108.test (exit status: 1)

FAIL: test_mathml_woluwe/test_109
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_109.test/expected.txt	2022-12-12 12:31:51.736096753 +0100
+++ /tmp/fooyfDiXo/output.txt	2024-06-04 13:15:05.218988718 +0200
@@ -1 +1 @@
-A*X/2"6Y:"6B
+AA_RX/2"6Y:"6B
FAIL test_mathml_woluwe/test_109.test (exit status: 1)

FAIL: test_mathml_woluwe/test_110
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_110.test/expected.txt	2022-12-12 12:31:51.740096764 +0100
+++ /tmp/foo6saQq2/output.txt	2024-06-04 13:15:05.242988860 +0200
@@ -1 +1,2 @@
-F*I:/J:<X>"7A*I:/J:<X>A*I"6#A:/J"6#A:<X>
+FA_RI:/J:<X>"7AA_RI:/J:<X>AA_RI"6#A:/J"-
+6#A:<X>
FAIL test_mathml_woluwe/test_110.test (exit status: 1)

FAIL: test_mathml_woluwe/test_111
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_111.test/expected.txt	2022-12-12 12:31:51.740096764 +0100
+++ /tmp/foo4VMkAS/output.txt	2024-06-04 13:15:05.266989002 +0200
@@ -1 +1 @@
-"2T\X*I:/M:"6X*J:/M::
+"2T\XA_RI:/M:"6XA_RJ:/M::
FAIL test_mathml_woluwe/test_111.test (exit status: 1)

FAIL: test_mathml_woluwe/test_119
=================================

warning: lou_getDataPath is deprecated.
warning: lou_getDataPath is deprecated.
Testing MathML to Woluwe braille.Diff: 
--- /usr/src/liblouisutdml/valami/liblouisutdml/tests/test_mathml_woluwe/test_119.test/expected.txt	2022-12-12 12:31:51.740096764 +0100
+++ /tmp/fooyp7p39/output.txt	2024-06-04 13:15:05.442990041 +0200
@@ -1 +1 @@
-"2T\%A*P:/N::"6%A*Q:/N:::
+"2T\%AA_RP:/N::"6%AA_RQ:/N:::
FAIL test_mathml_woluwe/test_119.test (exit status: 1)

Until this tests are not adopted the new changed tables with affects the dutch virtual dot feature change with released the Liblouis 3.30 version, impossible to increase Liblouis version number to 3.30 version, except not marks xfail flags the affected 18 testcases.
But the xfail markup is a temporary workaround only. :-):-)
Now have a dedicated maintainer the voluve tests in LiblouisUTDML?
Usual this tests are using I think the lbu_files/wiskunde.sem semantic action file and following tables, based from liblouis.ini the voluve tests main directory:

literarytextTable nl-unicode.dis,../../lbu_files/wiskunde.ctb,braille-patterns.cti
mathtextTable     nl-unicode.dis,../../lbu_files/wiskunde.ctb,braille-patterns.cti
mathexprTable     nl-unicode.dis,../../lbu_files/wiskunde.ctb,braille-patterns.cti
editTable en-us-brf.dis,braille-patterns.cti,../../lbu_files/wiskunde_edit.ctb

Bert, have an ydea the fix related? If I remember right you known the voluve tests hierarchy (sure better known with me). :-):-)
Samuel Don't like the mark xfail the failing tests type fixes, and will be releasing future a Liblouis version update to the 3.30 version the Debian Backports repository. :-):-)
And, a bonus:
Lot of changes happened in Liblouis side the 2023 year with based the Liblouisutdml 2.12.0 version, perhaps will be good if we doing future a fresh Liblouisutdml 3.13.0 release with contains all great new Liblouis related tables and languages (turkish table updates, new 3.30 version landed tables, etc), perhaps in jul or perhaps after the september publicated Liblouis 3.31 release.
I welcome helping this tasks related with not coding way, because me not have detailed C code experience, and active using Liblouisutdml. :-):-)

Attila

@egli
Copy link
Member

egli commented Jun 5, 2024

@bertfrees what should we do here? These tests fail almost with every release (as the dutch tables change). Are these tests useful?

@bertfrees
Copy link
Member

bertfrees commented Jun 5, 2024

Are the tests useful? There's no question about that! (Unless they are disabled with xfail.) Is it annoying that the test suite fails with almost every Dutch table update involving virtual dots? Yes. Is it annoying that Liblouisutdml is pretty much unmaintained and that it hinders smooth releasing of Liblouis in Debian? Hell yes!

That said, it should be relatively easy to fix. It's probably just some character that is used in wiskunde.sem (some bracket or underscore or something) that has gotten a different virtual dot. It breaks easily also because of the way the Liblouisutdml code is written. It could be improved a lot.

@hammera
Copy link
Contributor Author

hammera commented Jun 5, 2024

Bert, what the expected output braille sequence the failing tests the new changed virtual outputs related with perhaps affecting wiskunde.sem?
wiskunde.ctb or perhaps wiskunde-edit.ctb table are affected, or only the semantic action file or an included another subtable in Liblouis?
For Liblouis commit 9b7b15997c56d0772ccd86294d344b22fa14b32a have following commit entry affects the tables/nl-chardefs.uti:

commit 9b7b15997c56d0772ccd86294d344b22fa14b32a
Author: Bert Frees <[email protected]>
Date:   Wed Apr 24 22:21:55 2024 +0200

    Add more virtual dots to superscripts and subscripts
    
    Virtual dots are added to the superscript/subscript indicators, as
    well as the characters that are lifted/dropped.

Very interesting, when I examining after this feature merge and master branch of Liblouis UTDML cooperation with have a regression LiblouisUTDDML the test entries or not, when I setted up LOUIS_TABLEPATH environment value my /usr/src/liblouis/tables directory, I not experienced failure testcases, only when I do a full Liblouis 3.30 package.
I doed wrong the previous test before official 3.30 Liblouis release comed out?
Unfortunatelly, very difficult to doing a rolling development Liblouis package for debian, but perhaps I future will be trying doing my PPA a packaging recipe and for example a liblouis-git renamed package if not have better way to early we detect this issues future. :-(:-(
In Arch Linux lot of a11y packages have installable development snapshot packages, for example orca-git. But me don't want using Arch (I used it all the time, when a rolling update killed my entire sound system on a virtual machine, it's a good thing that I didn't use it as a production system). :-):-)
Anyway, for the future, I will solve this set of packages with daily updates for my own system, so that we can notice these problems in time, so that these phenomena do not have to be discovered afterwards.
If there is no capacity to fix these tests now, should I temporarily put them in xfail state and write a comment in the final commit with a list of tests, referring to this bug ticket, so that we don't forget to fix these tests later?

@egli
Copy link
Member

egli commented Jun 5, 2024

If there is no capacity to fix these tests now, should I temporarily put them in xfail state and write a comment in the final commit with a list of tests, referring to this bug ticket, so that we don't forget to fix these tests later?

@hammera can you look at the test results and see if you can fix them easily? Maybe just add the virtual dots to the expected results?

@hammera
Copy link
Contributor Author

hammera commented Jun 5, 2024

Ok Chris, I welcome trying investigate and fixing this issue if I get a little technical help, because I am not known detailed yet this tests how depending dutch virtual dots related with this tests, but I welcome learning new techniques. :-):-)
I need add proper virtual dots the semantic action file, or the test case entry expected.txt file?
Bert, in Liblouis side what subtable affecting this 18 tests? The nl-chardefs.uti file, or I need search the proper virtual dots the braille-patterns.cti file?
Dutch Braille with math related you use only between a and f characters the virtual dots related, or use the dot9 too?

Attila

@bertfrees
Copy link
Member

I can look at it maybe in the weekend.

@bertfrees
Copy link
Member

I think changing the underscore from 4569 to 456a might already fix it.

@hammera
Copy link
Contributor Author

hammera commented Jun 5, 2024

Very thank you Bert.

hammera added a commit to hammera/liblouisutdml that referenced this issue Jun 5, 2024
In dutch table changed the underscore virtual dot from 4569 to 456a with Liblouis 3.30 release.
Without this change follow up in Liblouisutdml, 18 voluve math tests are failing.
Fixes issue liblouis#106

Signed-off-by: Attila Hammer <[email protected]>
hammera added a commit to hammera/liblouisutdml that referenced this issue Jun 5, 2024
In dutch table changed the underscore virtual dot from 4569 to 456a with Liblouis 3.30 release.
Without this change follow up in Liblouisutdml, 18 voluve math tests are failing.
Fixes issue liblouis#106

Signed-off-by: Attila Hammer <[email protected]>
@hammera
Copy link
Contributor Author

hammera commented Jun 5, 2024

Bert, thanks the help! Works great (in Liblouis 3.30 only of course, Liblouis 3.29.0 not compatible the replacement). :-):-)
I opened Pr #107 with future fixing this issue, but now the sanitizer check and an another check failing.
Just a moment, I trying a magic thing, change the PR the Liblouis stable version number from 3.29.0 to 3.30 (I forgotted this). :-):-)

Attila

@hammera
Copy link
Contributor Author

hammera commented Jun 5, 2024

Bingo, version number increase the four affected files resolve the #107 check workflows related errors.
After the #107 merge this issue I think automatically closing.

Thanks the help the fix related.

Attila

@sthibaul
Copy link
Contributor

sthibaul commented Jun 5, 2024

Unfortunatelly, very difficult to doing a rolling development Liblouis package for debian

Why so?
It's a matter of somebody taking the time to upload snapshots to the experimental repository, like is done for e.g. gcc, llvm, etc.

@hammera
Copy link
Contributor Author

hammera commented Jun 5, 2024

Samuel, technical some way possible doing a day basis updated liblouis-bin-git style named package with automatically updates day by day in experimental repository when happens a change the main Liblouis repository the master branch?
So, for example liblouis-bin named with liblouis-bin-git, liblouis-data named with liblouis-data-git, etc., etc., etc.
Of course need handling conflicts with normal Liblouis related packages, and need handling perhaps provides and breaks fields.
So, I would like similar possibility with Arch doing with Orca main branch the orca-git package in Aur.
This package doing in Debian side need a Debian uploader permission, or anybody have possibility do a master branch Liblouis git based package upload the experimental repository?
How can syncing always the build system from the Github repository automatically (similar with Launchpad possible doing in Ubuntu syde the package recipe the git-build-recipe way)?
So, if anybody want using always the development fresh Liblouis version during development cicle with apt update method, this is a good possibility future if technical possible doing this in Debian.

Sorry the wrong english comment if difficulter understandable, but you sure understand what can I would like.

Attila

@sthibaul
Copy link
Contributor

sthibaul commented Jun 5, 2024

doing a day basis updated liblouis-bin-git

Daily uploads to experimental would be frowned upon :) In that case a PPA would be preferrable.

for example liblouis-bin named with liblouis-bin-git

I don't think we want to rename the packages (lots of reasons including package/files exclusions that will be hard to sort out). Just appending the version number with the git date should be fine.

I would like similar possibility with Arch doing with Orca main branch the orca-git package in Aur.

Similarly, a PPA with appended versions can be fine for that.

This package doing in Debian side need a Debian uploader permission,

For uploads yes, but for a PPA no.

@hammera
Copy link
Contributor Author

hammera commented Jun 5, 2024

Ok Samuel, everithing clear. In My Launchpad PPA I will doing this.
In Debian codename related possible creating a PPA (for example Bookworm) in Launchpad, or I use for example with 24.04 code series in Ubuntu to provide compatibility for Bookworm?
Oldest years Launchpad not provide possibility to create PPA for Debian, but possible this is already changed.

Attila

@sthibaul
Copy link
Contributor

sthibaul commented Jun 5, 2024

I don't know exactly how PPA works in ubuntu, but you can create a PPA on a mere webserver, you just need to build the package in the various targetted debian distributions, put them in separate directories, and run a few commands to generate the metadata

@hammera
Copy link
Contributor Author

hammera commented Jun 5, 2024

I known for example with Reprepro (tryed with local system), but the interesting thing how can possible doing the automatic architectures depending builds for example i386, amd64, armhf, and ARM64 (this is the four typical architecture I think with blind users using) a mere web server when the Github Liblouis repo happens a change for example the master branch.
Because I not known yet this special techniques with Debian related detailed, now using normal free Launchpad PPA and Ubuntu codenames. :-):-)

Attila

@egli
Copy link
Member

egli commented Jun 6, 2024

Fixed in #107

@egli egli closed this as completed Jun 6, 2024
@sthibaul
Copy link
Contributor

how can possible doing the automatic architectures depending builds for example i386, amd64, armhf, and ARM64

By first installing the qemu-user-static package, one can then create foreign chroots with debootstrap and run builds&tests in it. All the programs run in qemu simulation so that's much slower, but for liblouis that should be fine enough.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants