Incorrect file uri

[Leonti]

Hi!
First of all, thanks for the great IDE engine, I believe this will be the future of Haskell IDE tooling.
I have a problem though, my project compiles and works fine, but I get errors coming from /private/var/folders/... folder, for example:

2018-05-0813:56:23.784628[
   ThreadId 5
]- <--2--{
   "jsonrpc":"2.0",
   "method":"textDocument/publishDiagnostics",
   "params":{
      "uri":"file:///private/var/folders/hm/3t_b50tj3ng1fj3g4p9zgcn02hg99c/T/ghc-mod12335/PayslipCalculatorSpec12334-0.hs",
      "diagnostics":[
         {
            "severity":1,
            "range":{
               "start":{
                  "line":4,
                  "character":0
               },
               "end":{
                  "line":4,
                  "character":33
               }
            },
            "source":"ghcmod",
            "message":"Could not find module ‘PayslipArbitrary’\nUse -v to see a list of the files searched for."
         },
         {
            "severity":1,
            "range":{
               "start":{
                  "line":6,
                  "character":0
               },
               "end":{
                  "line":6,
                  "character":42
               }
            },
            "source":"ghcmod",
            "message":"Could not find module ‘PayslipCalculatorFixtures’\nUse -v to see a list of the files searched for."
         },
         {
            "severity":1,
            "range":{
               "start":{
                  "line":7,
                  "character":0
               },
               "end":{
                  "line":7,
                  "character":27
               }
            },
            "source":"ghcmod",
            "message":"Could not find module ‘Test.Hspec’\nUse -v to see a list of the files searched for."
         },
         {
            "severity":1,
            "range":{
               "start":{
                  "line":8,
                  "character":0
               },
               "end":{
                  "line":8,
                  "character":38
               }
            },
            "source":"ghcmod",
            "message":"Could not find module ‘Test.Hspec.QuickCheck’\nUse -v to see a list of the files searched for."
         },
         {
            "severity":1,
            "range":{
               "start":{
                  "line":9,
                  "character":0
               },
               "end":{
                  "line":9,
                  "character":32
               }
            },
            "source":"ghcmod",
            "message":"Could not find module ‘Test.QuickCheck’\nUse -v to see a list of the files searched for."
         }
      ]
   }
}

As you can see the file path is file:///private/var/folders/hm/3t_b50tj3ng1fj3g4p9zgcn02hg99c/T/ghc-mod12335/PayslipCalculatorSpec12334-0.hs
Also when I have real errors in my files the files they are not displayed in files themselves, but rather in those temporary files in /private/var/folders/..
I experience it both in Atom and VSCode, this is a screenshot from Atom

The cloned version is 53cd497168d83759a9cc25cff98a3134be0906c2 and it's installed using:

$ git clone https://github.com/haskell/haskell-ide-engine --recursive
$ cd haskell-ide-engine
$ stack --stack-yaml=stack-8.2.2.yaml install

Let me know if you need more information.

Cheers,
Leonti

[alanz]

What operating system are you using?

[Leonti]

It's OS X:

ProductName:	Mac OS X
ProductVersion:	10.11.6
BuildVersion:	15G31

[cideM]

I have the same issue on macOS High Sierra 10.13.x, in Visual Studio Code. I think it's solved temporarily by rebooting (I'll try to verify this). I've also ran into cases where clicking any of the error messages in one of those temporary files would open said file and also create a new file but with number at the end incremented by 1.
I am using the same HIE version as the OP and tried recloning and rebuilding.

[wz1000]

This might have been (re)introduced by the new URI handling changes to haskell-lsp. Can someone confirm if this issue affects afb99e6 or prior?

#545
haskell/lsp#76

[wz1000]

It would be helpful if the full path of the original file could be provided.
Also, is there anyone not on OSX experiencing this issue?

[Leonti]

I installed it at home on my Ubuntu 17.10 and I don't have this issue.
Very impressed with the engine - very fast on hover, hlint, docs, jump to definition works perfectly without installing tons of different plugins and packages. Implementing it as a language server was the right choice!
Thanks so much!

Seems like an OSX-specific issue though.
Will try to roll back to an older version tomorrow at work and see if it solves the issue.

[wz1000]

@Leonti could you provide the full path of the original file you were facing this issue with?

[cideM]

Here are the paths on my machine. Everytime I open

/Users/yuuki/private/haskell/hffp/chapter-exercises/src/Ch12.hs

in Visual Studio Code it creates a new tmp file. Right now

/private/var/folders/lt/cxg1b3t933x6xkdx12gl5vrr0000gn/T/ghc-mod24952/Ch1224951-4.hs

Close -> Open again and we have

/private/var/folders/lt/cxg1b3t933x6xkdx12gl5vrr0000gn/T/ghc-mod24952/Ch1224951-5.hs

The old tmp files are deleted whenever a new one is created so that when I now click errors reported for *-4.hs VSC says file not found. Click the current one *-5.hs opens the tmp file, not the actual file **/Ch12.hs

[Leonti]

My path was:
/Users/leonti.bielski/property-testing-demo/payslip-property-testing-haskell/test/PayslipCalculatorSpec.hs
and
/Users/leonti.bielski/property-testing-demo/payslip-property-testing-haskell/test/PayslipArbitrary.hs (for the second issue where errors are detected, but they refer to a temporary file)
My issue matches the one that @cideM describes

[wz1000]

Hmm, the paths look fairly straightforward, so there shouldn't be any weird URI decoding/encoding issues. I'll try to make a branch of HIE that logs all of this properly so I can see what's going wrong. Unfortunately I don't have an OSX machine on hand, so you guys would have to test it.

[Leonti]

Would be happy to test!
I will also try to see if I can find a commit from which the issue started to appear.

[zekzekus]

i am also on macos and having probably same issue in a different way. when i check the logs i see things like [Error] Got error while processing diagnostics: <no location info>: error:^@ module ‘seyler-0.0.0:Main’ is defined in multiple files: /var/folders/qx/fdvs1_qd5b7ch8y2j5yj6spw0 000gp/T/ghc-mod4644/Main4643-0.hs^@ /var/folders/qx/fdvs1_qd5b7ch8y2j5yj6spw0000gp/T/ghc-mod4645/Main4643-04643-1.hs

Also when i go to definition of a function i find myself on file with the temp path like above. And this one occurs only if the function on the same file. When going from another file it seems OK.

right now i can't supply detailed logs but i will as soon as possible.

[wz1000]

Can somebody use wz1000@052e1bc and provide me debug logs?

Run hie with

hie --lsp -d -l /log/file/here

and upload the log file to gist or pastebin or some such service

[nrskt]

I don't have this problem using macOS High Sierra.

{
	"resource": "/Users/xxxx/Documents/project/haskell/xxxxxxx/src/Sample.hs",
	"owner": "Haskell HIE (xxxxxxx)",
	"code": "undefined",
	"severity": 8,
	"message": "• Couldn't match type ‘Int’ with ‘[Char]’\n  Expected type: String\n    Actual type: Int\n• In the expression: i * 2\n  In an equation for ‘problemFunction’: problemFunction i = i * 2",
	"source": "ghcmod",
	"startLineNumber": 5,
	"startColumn": 21,
	"endLineNumber": 5,
	"endColumn": 26
}

environment as below

OS

ProductName:	Mac OS X
ProductVersion:	10.13.1
BuildVersion:	17B1003

VSCode

VSCode Version: 1.23.0

hie

Version 0.1.0.0, Git revision f526f9761dff62990befaf94ea8a21443d7463a1 (1223 commits) x86_64 ghc-8.2.2

[wz1000]

@nrskt Ok, the commit you are using is almost a month old. So we know that the issue is recently introduced.

Can anyone on OSX do a bisect to figure out which commit introduced this?

[zekzekus]

@wz1000 i checked out the commit you mentioned and here is the log file: https://gist.github.com/zekzekus/a1d9bd37fb592492bf58f314137b6e10

bisecting is taking too much while rebuilding. if i could know a working commit it would be much easier. if log's doesn't work i can try to bisect anyway.

[wz1000]

@zekzekus Could you test if changing line 60 in haskell-ide-engine/submodules/ghc-mod/core/GhcMod/FileMapping.hs from

  cfn <- getCanonicalFileNameSafe from

to

  cfn <- liftIO $ canonicalizePath from

fixes the issue?

[alanz]

We were requested to use makeAbsolute instead of canonicalizePath, for windows users. on HaRe. See RefactoringTools/HaRe#67

[zekzekus]

@wz1000 i observe same behaviour. here is the log file with the change you mentioned.

https://gist.github.com/zekzekus/5965ce24f9bc448a2cbd9f4318524a5c

[wz1000]

Ok, so I need more logs: wz1000@052e1bc

[Leonti]

I can confirm that f526f9761dff62990befaf94ea8a21443d7463a1 works properly for me
I tried bisecting, but I can't compile some of the commits with this error:

Stack looks for packages in the directories configured in the 'packages' variable defined in your stack.yaml
The current entry points to /Users/leonti.bielski/haskell/haskell-ide-engine/submodules/brittany/ but no .cabal file could be found there.

I did do git submodule update before trying to compile with stack --stack-yaml=stack-8.2.2.yaml install, ./submodules/brittany/ was still empty

[Leonti]

Here is my debug output with wz1000/haskell-ide-engine@052e1bc:
https://gist.github.com/Leonti/2f8f6af88114554dae0850ca4b08e902

This is the interesting bit:

018-05-09 12:06:08.726898 [ThreadId 5] - <--2--{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"uri":"file:///Users/leonti.bielski/myob/property-testing-demo/payslip-property-testing-haskell/test/PayslipArbitrary.hs","diagnostics":[]}}
2018-05-09 12:06:08.727022 [ThreadId 5] - <--2--{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":1,"message":"Got error response:{\"error\":{\"code\":-32601,\"message\":\"Unhandled method client/registerCapability\"},\"jsonrpc\":\"2.0\",\"id\":0}"}}
2018-05-09 12:06:09.319966 [ThreadId 7] - srcSpan2Loc: mapped file is /var/folders/hm/3t_b50tj3ng1fj3g4p9zgcn02hg99c/T/ghc-mod36475/PayslipArbitrary36474-0.hs
2018-05-09 12:06:09.320124 [ThreadId 7] - reverseMapFile: mapped file is /private/var/folders/hm/3t_b50tj3ng1fj3g4p9zgcn02hg99c/T/ghc-mod36475/PayslipArbitrary36474-0.hs
2018-05-09 12:06:09.320171 [ThreadId 7] - reverseMapFile: original is /private/var/folders/hm/3t_b50tj3ng1fj3g4p9zgcn02hg99c/T/ghc-mod36475/PayslipArbitrary36474-0.hs
2018-05-09 12:06:09.320243 [ThreadId 7] - reverseMapFile: Canonicalized original is /private/var/folders/hm/3t_b50tj3ng1fj3g4p9zgcn02hg99c/T/ghc-mod36475/PayslipArbitrary36474-0.hs
2018-05-09 12:06:09.320275 [ThreadId 7] - srcSpan2Loc: Original file is /private/var/folders/hm/3t_b50tj3ng1fj3g4p9zgcn02hg99c/T/ghc-mod36475/PayslipArbitrary36474-0.hs
2018-05-09 12:06:09.359234 [ThreadId 7] - setTypecheckedModule: Didn't get typechecked module for: "/Users/leonti.bielski/myob/property-testing-demo/payslip-property-testing-haskell/test/PayslipArbitrary.hs"

[Leonti]

@wz1000 I tried changing to cfn <- liftIO $ canonicalizePath from in haskell-ide-engine/submodules/ghc-mod/core/GhcMod/FileMapping.hs, but it didn't help :(

[OnlyMyRailgun]

@wz1000 I had this problem on OS X. After I revert to f526f9761dff62990befaf94ea8a21443d7463a1 mentioned by @nrskt, the problem has been resolved.

[wz1000]

OK, I seem to have figured out the problem. I added a canonicalizePath in the wrong place in my original fix.

The issue should be fixed by alanz/ghc-mod#9

Thanks for all the help and patience. You can test out the change by

$ cd haskell-ide-engine/submodules/ghc-mod
$ git remote add wz1000 https://github.com/wz1000/ghc-mod
$ git fetch wz1000 ghc-8.4-hie
$ git checkout ghc-8.4-hie

[wz1000]

The real issue seems to be that we lost this line https://github.com/alanz/ghc-mod/blob/47e200a728a575f407ee6f9893d9a1e77b1b5325/core/GhcMod/FileMapping.hs#L52 somewhere along the line

[Leonti]

@wz1000 Thanks a lot!
I can confirm that latest master(08b90ccbce5925648721f9ec90ce236302f586b9) works properly 👍

Thanks again for the awesome tool!