Updates for 64-bit and Objective-C 4

[GoogleCodeExporter]

Sorry for the lack of focus in this post (should it maybe be a Wiki page 
instead?), but it addresses a lot of different issues. 

Apple's Objective C runtime, GHC and Cabal have all changed quite a bit since 
the last time this project got much attention.  This is mainly just an 
experience report on some recent work I've done updating HOC to work with the 
latest versions of each of these.

The work in progress is currently available at https://github.com/mokus0/hoc.  
I'm using git because I find it so much more convenient than subversion.  Once 
it's all working again, I can push it back into SVN if that is preferred, or 
I'd be just as happy to take over managing the project if the current managers 
would like to move on, in which case I would probably move it officially to 
Github unless there's a strong community preference to keep it here.

There are several changes so far, with more to come:

- It builds with far fewer Cabal hacks, and the one remaining hack can easily 
be eliminated if we are willing to just abandon support for pre-ObjC-2 on Mac 
OS, which I think is a reasonable thing to do.  I think from an 
ease-of-building standpoint it is pretty close to ready for Hackage.  Getting 
to this point involved splitting it into two packages though - all the FFI 
imports are now in a separate "HOC-cbits" package.  This was necessary because 
GHC loads all imported modules to support template haskell, and this is the 
easiest way to get the "c bits" loaded.

- It runs on x86-64.  This was by far the hardest part, and also still the most 
delicate.  Passing of structures in the ABI is poorly specified (IMO) and 
Apple's compiler follows it even more poorly.  Currently it uses a simple 
threshold that happens to get most cases right, but this definitely needs some 
more thought.

- It runs on Mac OS 10.7.  This required a few hard-to-track-down but minor 
changes.  Most notably, "id" values are not necessarily pointers and 
ffi_closures need special care when allocating to make sure they come from 
memory that doesn't have execution protection.

- The test suite is now configured to run via "cabal test".  This means that 
building requires cabal 1.9.2 or newer.

- base < 4 support has been dropped.  If necessary it wouldn't be hard to add 
back later.

Other work planned:

- More testing and cleanup of my changes.  The code as it stands now will not 
work properly when compiled 32-bit due to a few quick fixes I haven't yet 
abstracted properly, but I plan to clean up that stuff tomorrow.  Also it 
appears that Objective C exceptions are not always being caught and marshaled.

- The interface generator does not work on recent Cocoa headers.  My first 
thought is to update it to use BridgeSupport metadata files.  I seem to recall 
from an earlier look at this idea that there was not enough information in them 
to support the interface generator.  If that turns out to be the case then I'll 
either change the generated module s or update the existing design to parse the 
newer files.

If anyone has any thoughts on either the changes made so far or changes that 
need to be made, I'd love to hear them.

Original issue reported on code.google.com by [email protected] on 28 Mar 2012 at 12:56

[GoogleCodeExporter]

Shoot, I realized too late that I entered this on the wrong google account so I 
couldn't set the ticket metadata properly.

Original comment by [email protected] on 28 Mar 2012 at 12:59

• Changed state: Started
• Added labels: Type-Review, OpSys-OSX, Component-Core, Component-Ifgen
• Removed labels: Type-Defect

[GoogleCodeExporter]

Great to see some activity here!

As the original author, I have no objection to you taking over managing the 
project for now; in fact, I'd be glad, as I haven't been doing any managing 
myself recently. I might get involved again at some point in the future, but 
the power belongs with the people who do the work, in this case. No objections 
against git either; the only thing I would miss if things move away from Google 
is the issue-tracker.

Original comment by [email protected] on 28 Mar 2012 at 12:49

[GoogleCodeExporter]

A note on the ffi_closure allocation code: mmap-ing a 4K page for each 
ffi_closure seems wasteful. I fear more work will need to be done at some point 
in the future to optimize this. 

Original comment by [email protected] on 28 Mar 2012 at 12:52

[GoogleCodeExporter]

Oh, sorry I forgot to mention the poor usage of mmap - thanks for reminding me. 
 Glad to see you've looked at the changes though ;)

I'm planning to implement a light-weight allocation scheme where the closures 
are allocated, say, pagesize / sizeof(ffi_closure) at a time and then we just 
take the next one when we need one.  Either that or if it's not too difficult 
(and available on gnustep) I might set up an NSAllocationZone or whatever it's 
called, but that seems like overkill since AFAIK we never actually free any 
ffi_closures.

Am I correct in thinking that if we wanted to we could even allocate them in 
batches of a few thousand without problems?  mmap should return memory that 
isn't actually backed until used, right?  Not that it's a big deal, but I'm 
pretty sure that sizeof(ffi_closure) doesn't usually divide the page size 
evenly, and an easy way of avoiding a bit of waste per page would be to just 
allocate a number of closures equal to the number of bytes per page.

Regarding the issue tracker - Google's is pretty nice, but Github does also 
provide an issue tracker - do you have any opinions about the relative merits 
of the two?

PS - an IRC discussion about this stuff broke out last night and a few other 
potential collaborators surfaced.  We've created #haskell-objc on freenode and 
a few interested parties are now lurking there.  If anyone is interested, feel 
free to join us (I'm "mokus" on IRC).

Original comment by [email protected] on 28 Mar 2012 at 1:16