Questions on integrating a new language for extensibility

[alexrp]

I've been thinking of writing some C# bindings for the Binary Ninja API, along with a native plugin that would load the .NET host and then further load (via the plugin manager API) any installed plugins written in C#.

My questions are:

1. Is a loader plugin in this fashion the best approach to adding C# support?
2. How might I go about providing a C# console, similar to the Python console?

[alexrp]

Ping; still looking for some input on this before I start implementing it.

[CouleeApps]

Generally speaking, a loader plugin is necessary for languages whose plugins may not compile directly to loadable modules. You need a native module as an entrypoint to the plugin api, although you can use the C FFI bindings from user code after that.

For example, the Python api uses a built-in native core plugin (pythonplugin.dll / .dylib / .so) which loads user python plugins from a directory. It handles versioning, resolving the user's selected python loader, virtualenv, etc, but generally just sets up a thread into which the binaryninja module is imported. The native plugin also searches the user plugins directory and imports all the various python files it finds in there. Apart from that, the python plugins do most of the interaction with the C FFI directly (via ctypes) and the native plugin is not necessary beyond that. I will caution you that the C FFI is not very well documented (apart from reading the python source) and subject to breakage as we generally don't guarantee ABI compatibility across versions. I would recommend picking one version (eg latest stable) and only targeting that. There are APIs to check ABI version support, which I would recommend using.

So for an implementation of C#, you will need a native plugin that loads the .NET runtime and any .NET plugins it finds in the user plugin directory (or another directory of your choice). From there, the plugins can use native methods to call out to the Binary Ninja C FFI (either through a library or directly), and can interact with any core systems through that.

I would not recommend trying to connect C# to the UI api, as that api is rather unstable and full of Qt nonsense. Even the python UI api bindings are brittle, and only work due to the magic that is Shiboken.

---

As for implementing a console scripting provider, I would recommend looking at the Python implementation. Generally speaking, there are a couple callbacks you have to handle relating to view/navigation updates, input/output handling, instance creation, and module loading (module loading is only used by the Plugin Manager). The UI should create provider consoles for you, just be careful about running commands on the main (ui) thread vs background threads.

I've heard of a couple people attempting to add console support for various other scripting languages, but I haven't actually seen any projects be completed. Support for them does exist in the product today, but the code paths are only tested for Python, as it's the only first-party scripting provider.

[alexrp]

Another question: Is the Python scripting plugin source available somewhere? I have some questions around how I should deal with initializing .NET plugins, and I think looking at the Python implementation would be a good start.

I guess mainly it boils down to whether CorePluginInit is called on the main thread or a background thread. I need to run a (managed) plugin host after initializing the .NET runtime. This host sticks around and takes care of loading (and possibly reloading) .NET plugins. Do I need to run this on a separate thread from where CorePluginInit is called? And if I do, am I going to run into threading issues when calling BN APIs? (I won't be binding the UI stuff in the API, so I'm only concerned about core APIs.)

[xusheng6]

No, the Python plugin source is not open-source. It is not very complex, though.

CorePluginInit is run on the main thread, so you would better return from it ASAP.

BN core APIs are thread safe. They take a lock internally if necessary. So for your case, there would not be any thread-safe issues.

[CouleeApps]

The Python plugin is basically just a glorified cpython wrapper for import binaryninja and adding the directories to sys.path. Almost all of the code interacting with the API is open source in this repo, including the scripting provider implementation. I would look at:

• https://github.com/Vector35/binaryninja-api/blob/dev/python/scriptingprovider.py
• https://github.com/Vector35/binaryninja-api/blob/dev/python/__init__.py#L144 (_init_plugins)
• _binaryninjacore.py (inside binja, not on github) / https://github.com/Vector35/binaryninja-api/blob/dev/python/generator.cpp for auto generated cffi bindings

[alexrp]

BN core APIs are thread safe. They take a lock internally if necessary. So for your case, there would not be any thread-safe issues.

In regards to the thread safety issues, I was thinking of this issue: #3769

Judging by that, it seems there are still some unanswered questions in this area. I'm just hoping I'll be okay if I stay away from UI/Qt stuff.

[CouleeApps]

Generally speaking, anything touching Qt/the UI needs to run on the main thread, that's just how UI frameworks work. The rest of the core apis are expected to be called from many threads (eg from the python console thread) and should have locking as appropriate. The trickiest places you may run into are callback functions, like BinaryDataNotification. Those are usually on background threads (called directly from whatever api is updating the bv). If you stay away from the UI though, you should hopefully be fine.