Update to latest Zydis version

[ghost]

Are there any plans on updating these bindings to the latest Zydis version with encoder support?

[flobernd]

cc @th0rex :-) ?

@johnined Zydis 4.0 is not officially released yet. We are planning to release it in the near future and afterwards the bindings probably will get updated as well.

[th0rex]

Yes I'll update it but I'm going to wait for the official release first (+ there might be some delay between the official release happening and me being able to start work on this).

[ghost]

Ah I see. Until then is it possible Zydis-C could be updated to official release 3.2.1?

[athre0z]

Not really worth the effort: the v4 release is just around the corner.

[oberrich]

@athre0z v4 is now officially released, do you want help updating the bindings or does the update_40 branch have all the necessary migrations already?

[athre0z]

I think I ported most of the decoder stuff on that branch, but the encoder isn't fully implemented yet. I still had some smaller uncommitted changes locally that I pushed now. The update_40 branch kind of escalated into me refactoring most of the library, so it's currently a bit chaotic. The zydis-c submodule on the branch also needs to be updated from the in-dev version that it is currently pinned to to the actual final version of v4.0.0 and will require a bunch of fixups for the changes that occurred between the two. I'll probably pick this up and continue working on it soon(TM) but won't give an ETA.

[oberrich]

I have bumped zydis-c version to 4.0.0, reviewed and (hopefully) accounted for all missing API changes in #31
The main thing that's missing now is a rust interface for the encoder (can't help you with that unfortunately).

[r3bb1t]

Hey. Still waiting for a 4.0 release. Encoder support is a really useful feature and i wounder how is the development going and will we see the 4.0 version in the near future?

[athre0z]

Not a lot of time to work on this right now, but I am making some progress every now and then when I have a few minutes of spare times. Hope to be able to release a first beta version of this soonish, though it will probably not have the encoder at first. My priority is to first update and polish up all the function that this library provided in 3.0 before focusing on the 4.0 additions.

[athre0z]

Ok, I think I'm reasonably happy with what we have now and merged the update_40 branch into master and published 4.0.0-beta.1 on crates.io. Tethering the binding's version to the Zydis version has the drawback that after the v4.0.0 final release we will be unable to do any breaking changes until Zydis v5 is out, which might be quite a while. I'm thus approaching this a bit more conservatively by first going through a few beta versions before setting the current public API in stone. Please feel free to test drive and let me know if you run into any issues!

[r3bb1t]

I've started porting my project towards zydis v4 and here is my short feedback:

• Library became more idiomatic, but it takes more lines of code to write things i have been writing before. Overall it's a good change, however, it makes my code base significantly larger.
• I want to store my decoded instructions in vector and use dedup and clone vector methods, but because of unsatisfied trait bounds i can't use them. Please, don't forget to implement them.
• I miss old DecodedInstruction, now it turned into Instruction<OperandArrayVec<MAX_OPERAND_COUNT>>. (I understand that it's needed to make code more rust idiomatic and distinguish between fully and not-fully decoded instructions.)

Here is my project: https://github.com/r3bb1t/vmp_analyzer. If you take a look at my code, you will understand why my code is gona become much bigger and less readable. (Yeah, it was not very good even before, but now will become even less readable 🌚) Here is an example

Overall, i really appreciate all your work, this disassembler is the best one i've seen and i sincirely hope that it will get more recognition just like it deserves. I just wanted to share my thoughts and tell a bit about my experience using this library.

P.s. Still waiting for full encoder support, that's the main reason i am moving from v3 to v4 🌞

[athre0z]

Thanks for the feedback!

I want to store my decoded instructions in vector and use dedup and clone vector methods, but because of unsatisfied trait bounds i can't use them. Please, don't forget to implement them.

I addressed this part here -- this should allow you to dedup (clone already handed by #33).

Edit: ah wait, I think this implementation is actually buggy: the array might be partially uninitialized memory. I'll fix this up in a moment / sometime tomorrow.

Edit 2: fixed in d16283a.

I miss old DecodedInstruction, now it turned into Instruction<OperandArrayVec<MAX_OPERAND_COUNT>>. (I understand that it's needed to make code more rust idiomatic and distinguish between fully and not-fully decoded instructions.)

This is less a matter of making things more idiomatic somehow and more a matter of the Zydis v4 changes where (also in the C library) operand decoding is now a separate / optional step. A lot of users simply don't need operands, especially hidden ones, and decoding operands is wasted cycles in these cases.

Instruction<OperandArrayVec<MAX_OPERAND_COUNT>> is quite the mouthful indeed: do you think that a adding type alias like

type FullInstruction = Instruction<OperandArrayVec<MAX_OPERAND_COUNT>>;

would improve the situation for you?

[athre0z]

Actually I just realized that I had already created these two aliases:

https://docs.rs/zydis/4.0.0-beta.2/zydis/type.AllOperands.html
https://docs.rs/zydis/4.0.0-beta.2/zydis/type.VisibleOperands.html

So you could actually do Instruction<AllOperands> already, which isn't too bad IMHO. I still added FullInstruction now in case people want something even shorter.

[athre0z]

Spent the weekend hacking up a high-level wrapper around the encoder interface and released v4.0.0-beta.4 with the first draft of this. See here as an entry point in the documentation.

I realized that X86 assembly is simple enough to be parsed by regular (non-procedural) Rust macros, so we can do stuff like:

let mov = insn32!(MOV EDX, 0x1234).encode();
let add = insn64!(ADD dword ptr [RDX + RSI * 8 + 0x1234], 1234).encode();

It'll still be difficult to JIT anything of meaningful complexity without an assembler type with label support, but I'm undecided on whether we should add this in the Zydis bindings because it needs a bunch of OS specific plumbing. Perhaps it's best to leave that for a separate crate.

[r3bb1t]

This 2 macros look really beautiful and in fact look even more cooler than what keystone offers. Also shows the beauty of rust macros. I would really love to use them, it will also save lots of code and make it cleaner.

I'm gona try to test all this in the next week and when I'll do it, i will give a feedback. I am really excited for the encoder support and I've been waiting for it so long.

P.s. Don't forget to update the zydis C library which is linked to the project. (Because of various bugfixes, including the wrong output of a function which gets the largest register encoding)

[Sewer56]

Note: For those experimenting with newer versions, you'll also need to run zydis-bindgen 😉 if you decide to upgrade zydis-c.

https://github.com/zyantific/zydis-bindgen/tree/master

In my case I needed encoder support, the encoder in current beta crate is not usable (it takes ownership rather than reference to encode buffer), so in addition to using upstream submodule, I figured I might aswell update. :P

Happy coding.

[athre0z]

I just released v4.1.0 as a stable: I think we can close this here. :)

I still want to do more usability improvements on the encoder API in the future -- some of which may end up being breaking -- but it seems that Zydis v5 will probably still happen within this year: lots of changes around Intel APX require some minor breaking changes in public C interface. I think the bindings are in a reasonably good state now, so I think that we can hold back any further API breaking changes until the not-so-far v5 update.

Thank you everyone with your patience around the v4 binding update: this took way too long. v5 will in all likelihood come with only minor breaking changes, so I expect this to go much quicker next time.