Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Obvious 1 strongly detected as a 7 #1

Open
Hexstream opened this issue May 8, 2024 · 6 comments
Open

Obvious 1 strongly detected as a 7 #1

Hexstream opened this issue May 8, 2024 · 6 comments

Comments

@Hexstream
Copy link

I'm not sure how actionable this is as a bug report, but I got this result here:

obvious-1-strongly-detected-as-7

@Hexstream
Copy link
Author

Even worse:

brutally-unrecognized-7

@Photosounder
Copy link

It's literally impossible to get a 6 recognised as a 6. This is the clearest 6 I could draw, the others are even more clearly recognised as 5s.
2024-09-11 13 39 54

@Hexstream
Copy link
Author

Hexstream commented Sep 11, 2024

Here are my 5 attempts at a 6,
I got 4 successes and 1 failure.

6-win-1.png

6-win-2.png

6-win-3.png

6-win-4.png

6-fail.png

@georgestagg
Copy link
Owner

georgestagg commented Sep 11, 2024

This is really not intended to be a state of the art computer vision model, instead as a small demonstration of using Fortran BLAS routines under WebAssembly.

The original Multilayer Perceptron model proposal is from 1958. So, yes, the model is not great. Probably nothing can be done other than replacing the model with something better.

For me it seems to work best on drawings closer to the scaled digits in the original MNIST dataset, where the numbers are fairly small and centred in the box. Probably it's overfitting to that source data, but it does not matter because model performance isn't really the point of the demo. The code tries to scale and crop digits, but I suppose line thickness might make a big difference.

Screenshot 2024-09-11 at 12 57 59
Screenshot 2024-09-11 at 12 58 06

Saying that, I don't find it "literally impossible to get a 6 recognised". So maybe something is going wrong on your machine.

Screenshot 2024-09-11 at 12 53 41

If it really is that bad, useful information would be the browser and OS you're using so I can try to reproduce.

@Photosounder
Copy link

Saying that, I don't find it "literally impossible to get a 6 recognised". So maybe something is going wrong on your machine.

If it really is that bad, useful information would be the browser and OS you're using so I can try to reproduce.

I see what the problem is, you guys draw 6 with a small loop at the bottom, I draw them with a bigger loop. When I draw a smaller loop that doesn't take up half of the vertical space of the glyph it does detect a 6. I suspect that it recognises the loop reaching the vertical centre as a defining feature of 5, I guess 6 was trained on small loops.

@Hexstream
Copy link
Author

Here are my 6 attempts at a big-looped 6, I got 3 successes and 3 failures,
but the last one is a very near miss.

big-looped-6-win-1.png

big-looped-6-win-2.png

big-looped-6-win-3.png

big-looped-6-fail-1.png

big-looped-6-fail-2.png

big-looped-6-fail-3.png

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants