Sass fails to read utf8 encoded files

[Wolvereness]

I outlined the problem on StackOverflow, copied here.

I'm running on windows (currently development mode). I've also tested with 2.8.1 (latest on maven) to the same problem.

I have a .scss file (encoded utf8) with this binary data (this is a specific example, there are more unicode characters in the file):

0xE2809C

Which is the character “ (left-quote).

The file gets included (by @import directive, not the asset-pipeline directive, but both methods fail to read it correctly) in another file, and when fetching the result, I get this binary data:

0xC3A2E282ACC593

Which are the characters “

Gradle build file has:

buildscript { 
    ...
    dependencies {
        ...
        classpath 'com.bertramlabs.plugins:asset-pipeline-gradle:2.7.2'
    }
}
...
apply plugin: "asset-pipeline"
...
dependencies {
...
    runtime "org.grails.plugins:asset-pipeline"
    runtime: 'com.bertramlabs.plugins:sass-asset-pipeline:2.7.2'
...
}

(not sure if there are any other relevant entries)

I made a reproduction case (latest default configurations): Wolvereness/Grails-Sass-Unicode-Test

Solutions that have not worked:

• @charset in either/both files
• -Dfile.encoding=UTF-8 on grails or the run-app command
  • Appears to be non-functional, see also No documentation on JVM parameters for CLI grails/grails-doc#455
• //=encoding utf-8 at the top of the file(s)

I did further investigation, and found that -Dfile.encoding=windows-1252 is being forced as a JVM parameter on my system. However, asset-pipeline should ignore that when the file explicitly specifies @charset or //= encoding, and plausibly because of the sass specification that will default to assume utf8 when not otherwise specified.

[davydotcom]

Since your using @import the directive will not be adhered to your in ruby code land now and that has different criteria for file loading... You really need to force the ruby runtime character encoding to utf-8

Sent from my iPhone

On Apr 26, 2016, at 4:11 AM, Wesley Wolfe [email protected] wrote:

I outlined the problem on StackOverflow, copied here.

I'm running on windows (currently development mode). I've also tested with 2.8.1 (latest on maven) to the same problem.

I have a .scss file (encoded utf8) with this binary data (this is a specific example, there are more unicode characters in the file):

0xE2809C
Which is the character “ (left-quote).

The file gets included (by @import directive, not the asset-pipeline directive) in another file, and when fetching the result, I get this binary data:

0xC3A2E282ACC593
Which are the characters “

Gradle build file has:

buildscript {
...
dependencies {
...
classpath 'com.bertramlabs.plugins:asset-pipeline-gradle:2.7.2'
}
}
...
apply plugin: "asset-pipeline"
...
dependencies {
...
runtime "org.grails.plugins:asset-pipeline"
runtime: 'com.bertramlabs.plugins:sass-asset-pipeline:2.7.2'
...
}
(not sure if there are any other relevant entries)

I made a reproduction case (latest default configurations): Wolvereness/Grails-Sass-Unicode-Test

Solutions that have not worked:

@charset in either/both files
-Dfile.encoding=UTF-8 on grails or the run-app command
Appears to be non-functional, see also grails/grails-doc#455
//=encoding utf-8 at the top of the file(s)
I did further investigation, and found that -Dfile.encoding=windows-1252 is being forced as a JVM parameter on my system. However, asset-pipeline should ignore that when the file explicitly specifies @charset or //= encoding, and plausibly because of the sass specification that will default to assume utf8 when not otherwise specified.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

[Wolvereness]

Sorry, I guess I didn't clarify. With or without @import it seems to be having the same issue, as evidenced by the reproduction case.

[davydotcom]

You might try the latest version of sass it uses Libsass now

[Wolvereness]

I'll give that a try next Monday; workstation is inaccessible this week.

[Wolvereness]

Okay, so I have some fascinating results... I took revision f4bbe67 and compiled it locally, as well as tested the publish 2.9.4 revision (I can't figure out which commit that was, but it was the latestest published as of me testing). I'll be using my test-case for the description.

Revision	Immediate File (index.css)	Imported File (style.css)	Immediate BOM (index.css)	Imported BOM (style.css)
2.8.1	Wrong output	Wrong output	Sass-failure in parser.rb	Sass-failure in parser.rb
2.7.4	Wrong output	Wrong output	Sass-failure in parser.rb	Sass-failure in parser.rb
2.9.4	Correct output	Wrong output	Functional	Thrown grails exception in jsass*
2.10.0 (f4bbe67)	Correct output	Wrong output	Functional	Thrown grails exception in jsass*

Here's a possibly relevant stacktrace (from the jsass stuff) from adding a BOM to the imported styles.css and removing the //=encoding UTF-8 line (although, adding that back only changes the error line). This is the asterisked stacktrace.

ERROR org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/].[grailsDispatcherServlet] - Servlet.service() for servlet [grailsDispatcherServlet] in context with path [] threw exception [Filter execution threw an exception] with root cause
io.bit3.jsass.CompilationException: Error: Invalid CSS after "∩": expected 1 selector or at-rule, was "∩╗┐@charset 'UTF-8'"
        on line 1 of styles.scss
>> @charset 'UTF-8';
   ^

        at io.bit3.jsass.adapter.NativeAdapter.compileString(Native Method)
        at io.bit3.jsass.adapter.NativeAdapter.compile(NativeAdapter.java:55)
        at io.bit3.jsass.Compiler.compile(Compiler.java:111)
        at io.bit3.jsass.Compiler.compileString(Compiler.java:61)
        at io.bit3.jsass.Compiler$compileString.call(Unknown Source)
        at asset.pipeline.jsass.SassProcessor.process(SassProcessor.groovy:49)
        at asset.pipeline.AbstractAssetFile.processedStream(AbstractAssetFile.groovy:164)
        at asset.pipeline.DirectiveProcessor.fileContents(DirectiveProcessor.groovy:307)
        at asset.pipeline.DirectiveProcessor$fileContents.call(Unknown Source)
        at asset.pipeline.AssetPipeline.serveUncompiledAsset(AssetPipeline.groovy:65)
        at asset.pipeline.AssetPipelineFilter.doFilterInternal(AssetPipelineFilter.groovy:174)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
        at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:112)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
        at org.grails.web.servlet.mvc.GrailsWebRequestFilter.doFilterInternal(GrailsWebRequestFilter.java:75)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
        at org.grails.web.filters.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:67)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
        at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:103)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:522)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1095)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:672)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:745)

[davydotcom]

got this fixed for new release