Generate source maps when running file directly too

[aseemk]

Extracting this request out of issue #166 as... requested. (Pun intended.) 🎤

I'd love to be able to run Streamline code directly, and still get source map support (e.g. for use with node-inspector).

This could be achieved by either generating the source maps in a temporary directory (instead of alongside the source files), or if it's possible, in memory directly.

(I didn't even know the latter could be a possibility until it was mentioned in a similar issue for CoffeeScript: jashkenas/coffeescript#3187 (comment) Does the source map comment format support embedding the source map directly maybe?)

/cc @goffrie — I would be grateful for your continued expertise on this matter. Thank you!

[spollack]

related to this, it would be great to be able to generate source maps not just when running via _node or _coffee, but also when using the register() api path.

[aseemk]

Indeed! That's actually my/our use case too. (_node and _coffee are just thin wrappers around register().)

[aseemk]

From jashkenas/coffeescript#3187 (comment), a quick workaround for this for .coffee files:

https://gist.github.com/benjie/9387284

No reason this workaround couldn't be used ._js and ._coffee files, too, right?

[goffrie]

It was originally supposed to be the case that you would get source-mapped stack traces when running with _node/_coffee. This stopped working at some point, but as of current master you can now use _coffee -m file._coffee to turn this feature back on.

However, this only stores the source maps in memory, so it won't work with node-inspector. It is possible to add a sourceMappingURL comment to the generated JS, so that node-inspector can access the source map. However, I'm hitting a problem here - since Streamline loads the compiled module under its original filename, there's no way to distinguish, in node-inspector, between the compiled and original files. This leads to the source display being incorrect (always displays compiled source, even when using original line numbers).
One way to fix this is to give the generated JS a different filename. But this changes semantics: __filename will use the new filename. In fact streamline's async stack traces use this, so they will look strange in this situation.
So I don't know what to do here. :(

[aseemk]

@goffrie:

as of current master you can now use _coffee -m file._coffee to turn this feature back on.

Cool! Would this work with the register() API too, if we set the sourceMap option to true? (I notice command.js also sets lines to 'sourcemap'; is that needed too?)

Tricky situation with the source vs. compiled files. One hacky idea that comes to mind is to create a copy of the source file somewhere temporary, for the purposes of the source map pointing to it. But then I imagine in node-inspector it'd look like your source files were in some temporary folder, which is no good either. Dang...

[goffrie]

It should work with the register() API (this is how I was originally using it), though I didn't test it. Try it :) I believe that it should be on by default - only requires lines be sourcemap, which is the default in underscored.js. The sourceMap option is only used in compile.js, so it shouldn't be needed.

[aseemk]

@goffrie: you're right for register(), but looks like the default is preserve if running through the _coffee command-line wrapper:

https://github.com/Sage/streamlinejs/blob/v0.10.8/lib/compiler/command.js#L8

Should that default be changed? Not sure. (cc @bjouhier)

[aseemk]

@goffrie:

One way to fix this is to give the generated JS a different filename. But this changes semantics: __filename will use the new filename. In fact streamline's async stack traces use this, so they will look strange in this situation.

I just realized: if stack traces take advantage of source maps, why can't they use the "original" filename from the source maps, just like node-inspector?

If they could, would there by any other downside to changing __filename? Could we simply document that when running e.g. /path/to/foo._coffee (both through the _coffee executable and via the require hook), __filename is always /path/to/foo.js? This almost has a nice consistency to it: the semantics of your code will remain the same even if you compiled the file manually.

[aseemk]

To clarify, I imagine this means that when we run or require /path/to/anything._coffee, we actually tell Node we're running /path/to/anything.js, even if we don't output /path/to/anything.js, and we point our source maps to the original /path/to/anything._coffee. Then node-inspector will work perfectly, and if the stack traces could be stack-trace-aware, they'd also look right.