Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for custom decorative stitch formats: .pmx, .pmu or .pmv #61

Open
mskressin opened this issue Jul 24, 2018 · 31 comments
Open

Comments

@mskressin
Copy link

Some Brother machines (such as the SE600 series and the Q series) allow the user to read decorative stitch format files and then sew stitches in the design specified in the file. These files have .pmx, .pmu or .pmv file extensions. Some other Brother machines (such as the Q series) additionally include a feature which allows the user to manually input stitch coordinates to create and save a decorative stitch in a .pmv format file It would be very nice if we could use the InkScape/InkStitch tool to take a design and convert it into a custom decorative stitch. This feature would make custom decorative stitches much easier to create and share. The following link describes this manual process for creating and inputting custom decorative stitches: http://support.brother.com/g/b/faqend.aspx?c=us&lang=en&prod=hf_inovnq1300eus&faqid=faqh00100482_000

@tatarize
Copy link
Contributor

Like draw a basic connected line thing in inkstitch and export it to the stamp, motif, decorative stitch formats? pyembroidery would just be sent the 5-25 stitches and just save it out? I know .pas files are pretty trivial and are basically the same thing. Hm. Sounds like the only thing that would be needed would be writing the formats.

@tatarize
Copy link
Contributor

Seems like I can do .pas files and .pmf, not found one of the other ones yet.

@tatarize
Copy link
Contributor

And I've coded this exact functionality before too. I have pattern stitches in my android embroidery app. The manual seems pretty down on the idea using this data from other sources. "In the Character/Decorative Stitch memory, ".pmv", ".pmx" and ".pmu" stitch data files can be used with this machine. ".pmu" or ".pmx" stitch data files can be retrieved; however, when the machine is used to save the file, it is saved as a ".pmv" stitch data file. Using data other than what was created using this machine or the machine which create ".pmv", ".pmx" or ".pmu" data file, may cause the machine to malfunction."

@tatarize
Copy link
Contributor

Tots worth the math, though, https://youtu.be/wPlq7Hu1pok

To do this I'd need some example files. I cannot imagine they are at all complex. Probably something like a series of integer coords.

@mskressin
Copy link
Author

I have some .pmv files I exported via the save to USB feature of my machine. They contain a copyright notice at the beginning so I would not feel comfortable posting them to a public space. Is your email address available anywhere?

@tatarize
Copy link
Contributor

Sure tatarize at gmail would be fine. Or hit the buttons on your machine to make some box patterns or something. That way you'd own the IP.

@tatarize
Copy link
Contributor

The first 100 bytes are header. This information is usually able to be disregarded but for a writer it matters. This is followed by the count of the stitches, then the length of the stitch block. After the stitch block there's 150 bytes of data. The first 98 is all identical and I dunno what it could mean. The last 52 differ only in the one of the columns. The other column (2 byte little endian values), goes up by 50s from 0 to 700. Pretty sure these are again coordinates but the included Y? values are monotonic. Because it's a graph of some sort. The other column is also monotonic 2 byte little endian values but they are not evenly spaced out.

@tatarize
Copy link
Contributor

For the reader I'm stuck at the encoding. Found the relevant stitches but somehow to fit it needs a constant new bias of Y and even then it's still broken. It's actually kind of weird but I think the locations are accounting for stitching forward. Adding a temporal skew makes it actually somewhat close. That said the writers might be unstable and that warning message seems ominus. But, it would rightly be trivial to add a stitch to the inkscape, edit it and save it out. That actually sounds awesome, even if it risks your machine turning sentient and killing your or whatever that warning is about.

@mskressin
Copy link
Author

The wording of the warning "Using data other than what was created using this machine or the machine which create ".pmv", ".pmx" or ".pmu" data file, may cause the machine to malfunction" is interesting, because of the phrase "or the machine". I chalked it up to a translation issue between Japanese and English thinking the intent was "or a machine". Assuming it is a language translation issue, it implies that files can be exchanged between machines. If this is true, would it not imply that what is needed to decode the file is either completely contained in the file or is somehow built in to every Bother machine which can read the files.

On the other hand, due to the lack of shared .pmv, .pmu or .pmx files on the internet coupled with the fact that Brother distributes stitches via the codes to be manually entered into the "My Custom Stitch" feature of some machines and not via a .pmv, .pmu or .pmx file, might confirm that the encoding may be unique to each machine. This might also explain the existence of more than one stitch file format. The warning also indicates that a machine is only capable of creating a .pmv file. This .pmv file format may be specific to a particular machine due to the need for some sort of internal calibration constant that you may have discovered. Perhaps a .pmx or .pmu file can be shared between machines and a .pmv cannot. Unfortunately I have not been able to find any of the files on the internet to even look at a .pmu or .pmx file.

@tatarize
Copy link
Contributor

If you're going to take a warning and then argue with the definition of "is" is, that warning doesn't mean much of anything to you.

The first 100 bytes of the file seems to be header. This comes in two parts the first 44 bytes contain text. This is divided into version info, and copyright. The version info is quite standard for Brother #PMV0001 it's a PMV file version 1. This implies the file isn't some amazing very specific type exclusive to this machine. They kind of standardize their stuff so all PMV files of that same version will have the same parts in the same order. This is followed by 56 bytes of either 00 or 01. These are quite likely to be settings. Like in the stitches, there's likely some settings you can set. Save a file, set that, save a new file. Odds are good your 00 turns into a 00. These are all the same except for the satin stitch. I have that seems to have other settings.

This header info is followed by the stitch data blocks.

The stitch data blocks contain the actual stitch data. The block is premabled with the number of stitches and the length of the block. This is almost always 2 * (number of stitches) since the stitch is totally just 2 bytes long.

The stitch blocks are laid one after another, and these terminate at a point where it says 0, 256 for the stitch and stitch length. This is true for the when the data contains multiple stitchdata blocks or not. It terminates with 0, 256. It doesn't provide the number of the stitch data blocks so it needs a terminal element. In hex this say 00 00 00 01, in little endian 16s that's 0, 256. This is followed by 16 bytes that read either:
"00 00 00 00 05 00 00 00 00 00 00 00 00 00 02 00"
"00 00 00 00 05 00 00 00 00 00 00 00 00 00 05 00"
I do not know what this means but if it's 16le like the rest it's the numbers 0,0,5,0,0,0,0,2 for some reason.

After these we have another 00 01 (256. I'm guessing that might be block end). But, the cusps are always weird. This is followed by 2 dunno values.

Then we have the steps. The first value, is another dunno, but the second is the number of pairs. Then we have the pairs of 16le values that montonically increase usually by the same amount. The first values go 0, 20, 40, 60... the second values go like: 0, 71, 143, 214, 286, 357, 429...

We then have another dunno value, then the size of the second steps. Note steps aren't the same size, they aren't equal to the size of thes stitch number either and they don't have the same values as each other. You can have different stitch values, steps1 and steps2, and often do.

The second have the first value go 0,50,100, 150... and the second like 0, 200, 400, 600, 800, sometimes 0, 100, 200, 300... Sometimes just 0, 8192 as the entire set of values.

The steps are followed by the number 18 (0x12) and then 16 bytes of zero. Which makes sense for an ending bit.

@tatarize
Copy link
Contributor

Most of this wouldn't matter much but the stitch values are the stitch values and I can't parse them. I would expect them to be properly similar to the things you enter on the screen. But, they aren't. I can't figure them out.

I watched a couple videos of such stitches being stitched and a few things become obvious. The sewing machine moves forward. This tends to be unlike an embroidery machine that just embroiders a place. The monogramming foot moves forward and doesn't move forward a constant amount. It jumps in small jumps and big jumps. Within the foot there's a tiny amount of wiggle room for the needle and it seems to go a little bit in all directions, though maybe it just moves forward and backwards with the actual pressure foot just goes side to side. That would make more engineering sense.

This sadly speeds up the sewing rather than show it in slow mo.
https://youtu.be/hAox3N4Kg-g?t=4m10s

And some custom stitch things have some other options. I wouldn't be shocked if the width controls were actually in the file.
https://youtu.be/mvBEntfEYTQ?t=2m29s

I'm guessing a few of the mysteries could be solved by changing the settings on a stitch and saving out a new file several times thereby identifying options like width, or whatnot on the machine and consequently into the meh pile. For the bytes.

Some more mysteries could be solved by strategically editing the file in hex and seeing what it does to the machine. Though that would start to violate that warning. We know where the stitches are, what happens if we change one of the stitches slightly, does that show up on screen... usually that's how such things are solved. But, without figuring out the coordinate system it's a bit like I've identified the various varieties of poop around the file, but can't figure out how the diamond works.

@tatarize
Copy link
Contributor

Ah, looking at a high res image of it sewn out.. and counting the stitches, is apparently really helpful.

https://amateurlibrarian.com/2011/08/15/sewing-machine-sampler/

The question of how do these 23 number pairs:
(2, 0),
(0, 25),
(62, 18),
(14, 18),
(14, 18),
(14, 18),
(2, 25),
(2, 0),
(62, 7),
(54, 7),
(54, 7),
(0, 0),
(10, 0),
(62, 25),
(54, 25),
(54, 25),
(0, 0),
(0, 7),
(62, 14),
(14, 14),
(14, 14),
(14, 14),
(2, 7)

Become this:
presented

Turns out to be a functionally harder question to how do those numbers become:
stitch

@mskressin
Copy link
Author

When you create a custom stitch with the "My Custom Stitch" feature on the Q series machines, you enter grid coordinates based on where the design intersects with the grid on the Brother Grid Sheet. For example:
image
The numbers you posted look like x and y grid coordinates. But what is not clear is the order you listed them. For that design, I would have expected something like:
(0,6)
(0,4)
(0,2)
(0,0)
(2,0)
(4,0)
(6,0)
and so on all the way around the design.

@tatarize
Copy link
Contributor

I would have expected that too. I looked for that. I checked all the other info that exists. Those values are the correct values. They seem to repeat at places that make sense for do the same stitch in the same direction, but I can't figure out the connection I would think they would be advanced presser foot and move needle side to side, but they don't seem to be. They seem to be such that the same number results in a stitch in that same direction the same distance. But, not like one number the same but that's true when both numbers are the same and clearly for points that are going in the same direction. Sort of like change in position rather than global but I wouldn't expect that for the needle given it goes back and forth. Sometimes the sequence goes 0, 7, 14 for the second element then when it should go backwards it goes 0, 25, 18 in ways that seem to indicate those should be monotonic. Is 25 = -7, and 18 = -14? I mean the design there is really half the design given in the manual and I have the actual needle penetrations (thanks to some librarian quilter), and I still can't figure out what the points have to mean.

I have a degree in computer science, I developed this back end by doing largely work exactly like this pulling pattern and understanding out of hex codes with subject knowledge, logical inference, engineering and skill. I can can write an entire embroidery file from scratch in binary in half a dozen file formats. And I think this simple stitch encoding has me beat. And I've literally programmed stuff that does exactly this, in my embroidery android app's pattern stitch feature. It just has me stumped.

@tatarize
Copy link
Contributor

And then there's all the other clues I have, like this stitch:

presented2

Comes out to:
(2, 18),
(3, 26),
(2, 18),
(3, 27),
(2, 19),
(3, 28),
(2, 20),
(3, 30),
(2, 21),
(3, 0),
(2, 23),
(3, 3),
(2, 25),
(3, 6),
(2, 28),
(3, 8),
(2, 30),
(3, 11),
(2, 1),
(3, 12),
(2, 3),
(3, 13),
(2, 4),

And you think aha! gotch now, but then after it starts using 0,3 like it's 3, 2.. What? How are those numbers equal? How do they make that kind of pretty satin curve. And if it's even width doesn't it mean the distance from 1, 12 is about the same as 3 to 13. Well okay, but then it starts doing 5,27,3, 25,2,24. What!? How is that even diameter satin stitching?!

... (3, 14), (2, 5), (3, 14), (2, 6), (3, 14), (2, 6), (3, 14), (2, 5), (3, 13), (2, 4), (3, 12), (2, 2), (3, 10), (2, 0), (3, 9), (2, 30), (3, 7), (2, 27), (3, 4), (2, 25), (3, 2), (2, 23), (3, 0), (2, 21), (3, 30), (2, 20), (3, 28), (2, 19), (3, 27), (2, 18), (3, 26), (0, 18), (3, 26), (0, 18), (3, 26), (0, 18), (3, 27), (0, 19), (3, 28), (0, 19), (3, 29), (0, 20), (3, 30), (0, 21), (3, 31), (0, 22), (3, 1), (0, 23), (3, 2), (0, 24), (3, 4), (0, 25), (3, 6), (0, 27), (3, 7), (0, 28), (3, 9), (0, 30), (3, 10), (0, 31), (3, 11), (0, 1), (3, 12), (0, 2), (3, 13), (0, 3), (3, 13), (0, 4), (3, 14), (0, 5), (3, 14), (0, 5), (3, 14), (0, 6), (3, 14), (0, 6), (3, 14), (0, 6), (3, 14), (0, 5), (3, 14), (0, 5), (3, 13), (0, 5), (3, 13), (0, 4), (3, 12), (0, 3), (3, 11), (0, 2), (3, 10), (0, 0), (3, 9), (0, 31), (3, 7), (0, 30), (3, 6), (0, 28), (3, 5), (0, 27), (3, 3), (0, 25), (3, 2), (0, 24), (3, 0), (0, 22), (3, 31), (0, 21), (3, 30), (0, 20), (3, 29), (0, 20), (3, 28), (0, 19), (3, 27), (0, 19), (3, 26), (0, 18), (3, 26)]

I took this up because I thought it would take me a few hours and be kind of a cool thing to have, use inkstitch to load the file up, edit it and save it out, constraining it so that it's going to be valid (50 point limit, make sure the needle isn't going to hit the presser foot, gold)... but nope. I don't even understand the coord system. I rewrote a affine transformation system into pyembroidery because I didn't want to use any other libraries and because the math was trivial for me. But, you give me 5->13 = 25->2 and I'm all like waaaaa? And I'm patting myself on the back for brilliant guesses like that graph paper isn't actually isomorphic clearly the time it takes for the needle to move up to the edge must make it slower and thus bends it. So the paper is ensure that when people enter a straight line they must nudge the extremes back a little. Aha! -- And then it's back to how is the same number seem like the same level and the same pair the same distance stitch and it feels like reading tea leaves. That needle penetration chart provided a lot of insight, not enough to solve anything but I felt so much more like this was cracking open and giving up all the secrets. Nope.

@mskressin
Copy link
Author

I very much appreciate you trying to solve this problem. I too thought, "how hard can this be". Surely the file just contains the coordinates similar to the way that you create the stitch via the "My Custom Stitch" feature available on some models of their machines. My next step is to contact Brother and see if they would be willing to release the format of these files. I don't have much hope they will agree. However, perhaps I can convince them that allowing the easy creation and sharing of custom stitches will only help them sell more machines. Especially since they don't sell the custom stitches to customers. Once again, not much hope, but worth a try. Thanks again for your help. You can go ahead and close this request if you wish.

@tatarize
Copy link
Contributor

Nah, the next step is to hope somebody cares and has that type of sewing machine with that particular feature. Then just ask them to map the format out. First program a stitch that goes to 1,0, then another stitch file that goes to 1,14 then another that goes to 14, 1 and then 1, 14. And export those files. Simple learning based custom stitches. See what they generate then do other where you know the exact values, like go 0,0, 1,1 2,2, 3,3, ... 14, 14 and just look for patterns in those while knowing precisely what the values mean.

That or come back to the problem later. The information is there. It's totally solvable. Leaving the issue is fine. Somebody might solve it. I probably will go all Ahab on it at some point. It's likely something stupid or some encoding thing I didn't consider. Obviously it's built for a machine not a human. Humans enter the numbers into the device and it puts the stitches there, but the device likely builds the files in a manner the machine can use.

Like the steps things at the end, seem like they are a property of the width and height. They will be something like the grid lines behind the stitch or something.

And there's always the option of hex editing one of the files in a particular way and see the screen and how it changed that. Odds are good the warning is because you might be able to program a stitch to do something nutty and hit the presser foot, but if you stitch like 5-05 locate the stitches (they will be around byte 102 (0x66) and find the stitch that must exist in there that basically jumps off to the side then back in a parallel fashion and then tweak that number then look at it on screen.

Odds are good since you're just changing the data rather than deleting anything it would be valid data throughout, you wouldn't even need to run it. Just see the onscreen image. Doing that is how ww, helped me map out basically all of .u01 format. I know what all the commands are because I just wrote up hex edited files that replaced one well known command with literally every possible other value it could be.

Just write the files with truncated auto ending steps and standard headers. I could make a writer right now that might even work. The problem is without knowing what the numbers mean I can't make it say anything cogent. I could likely write a script to generate a random 5 character long pattern. Though, I would just be pulling the values from a hat.

If you wanted to give that a shot I could send you a file with all five 0,0 values and five 2,2 values. Might cast some light on what a repeated value means. Or if something like all 14, 14 means the same or whatnot.

@mskressin
Copy link
Author

Sure. I can't imagine it would cause any damage to the machine, since I can view the stitch without actually sewing.

@tatarize
Copy link
Contributor

tatarize commented Jul 26, 2018 via email

@tatarize
Copy link
Contributor

tatarize commented Jul 26, 2018 via email

@tatarize
Copy link
Contributor

6-1

6-1

So the pairs are the first value is forward amount and the second value is position within the foot. The position in foot is a 5 bit signed number. Ranging from 15-0 and -(32 - y) for values greater than that. So anything that doesn't have a negative value for the forward seems pretty solved and I'd guess the x values must also be similar but maybe with different bit ranges.

@tatarize
Copy link
Contributor

tatarize commented Jul 29, 2018

7-5

7-5

@tatarize
Copy link
Contributor

5-13

And that's 5-13. Hence the trouble figuring it out, that edge moving stuff isn't actually built into the format, but it is often built into the stitch. Not really sure if there's a native unit to use with these things though. And they are notably quite tiny. The spiky stuff in places is inkscape making different caps.

So in the end the stitch are stored in series of pairs located around 64, in with 4 bytes of size data. Then the pairs are 6 bit position forward and back and 5 bits position side to side. Where the forward and back parts are always relative and the side to side is always global. QED.

@tatarize
Copy link
Contributor

tatarize commented Aug 2, 2018

It's a bit tentative but it writes the stitches too now. It can read and write them. I might have been a bit too restrictive with the requirements but if the height exceeds +- 16 it aborts or if there's more than 64 stitches it aborts. But, for most things that's going to be more or less correct. I could certainly reband all the stitches into that +16 by -16 set of values. So regardless where they were they'd end up getting shoved into that and then only process the first 64 stitches or so. It would at least allow all patterns to be correct even if they were originally an embroidery. Rather than breaking and causing a zero byte file.

But, that's basically it for my end. It reads and writes that format.

I still have a few odds and ends to finish before I'll declare another version (still in the 1.2.x tree). But, after that, in theory, if you could get Lex to add the hooks you really could edit stitch files in inkscape. There's also a few other readers that inkstitch currently doesn't process even though they are part of pyembroidery. For example writing .u01 files and reading a bunch more fairly rare formats.

You could also currently manually enter the values, that is after all what those other machines currently allow. Or if you want to try it just give me the values, and I'll process it for you, seeming a pretty good end to end test. And the file could totally be just posted here as there'd be no other copyright on it.

@tatarize
Copy link
Contributor

tatarize commented Aug 5, 2018

Solving the Steps (really lookup charts for the variations in length and width) gives units to the stitches. This requires some rewriting because they have real world size conversions and that's 1/10th mm internally so with unit conversion. These stitches are not unmoored. Applying a scale factor means we could convert any embroidery to a stitch, it would just be scrunched and truncated. The width is solved for how it works. The length is still unknown and maybe just forced to chart as a constant.

@taylorsamy
Copy link

Has there been any progress on this feature? I would love to have it.

@zmeutz
Copy link

zmeutz commented Apr 4, 2020

Any news about this? any software that can convert from pes to pmu?

@lexelby
Copy link
Member

lexelby commented Apr 5, 2020

Sorry, there hasn't been any progress, I'm sorry to say. Without having a detailed specification of the file format it's difficult to proceed.

@PlantLily
Copy link

I'm not sure if this will be of any help, but I found This which details the PMV structure and functionality. Hope this helps!

@lexelby
Copy link
Member

lexelby commented Jul 30, 2020

Thanks! Looks like Tatarize wrote that at some point during or after the discussion above. I have to be honest: I find his writing at times kind of difficult to decipher. I'd have to study that wiki page to see if I can make sense of it.

In any case, thanks for the pointer!

@lexelby
Copy link
Member

lexelby commented Jul 30, 2020

Ah! Looks like Tatarize wrote a reader/writer for his fork: https://www.github.com/EmbroidePy/pyembroidery/tree/master/pyembroidery%2FPmvWriter.py

Our repos have diverged a bit since we parted ways, so it may take a bit more than just dropping that file in, but at least we're closer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants