Guide – Expressions: advanced usage

[rochoa]

See #532 (comment).

[davidmanzanares]

I don't know if this is the best place for this, but since I was trying to work on this issue and it's particularly related to the linked comment I'll write here. I can open a new issue if you think it's better.

After a general review of the guides I think I have found some problems:

• Minor issues (subjective issues). For example, see my PR: Improve getting-started #906
• The order of the guides. For example, I think that "Introduction to interpolation" shouldn't be before introducing ramp, which is done in the last parts of https://carto.com/developers/carto-vl/guides/introduction-to-styling/ I think we already have good content in general, but it the order is unexpected and the guides seem to have been written in a random order by different persons (which is how it was done 😂)
• Mixing advanced with basic stuff without notice. As an example read this (from the first parts of the Animation guide):

The convention in CARTO VL is that 0 represents the boolean value false, (the absence of or an off state), while 1 represents true (the presence of or an on state). When the expression assigned to filter (animation in our case) has the value 0 for a given feature, that feature will be filtered-out and not be shown. If the value is 1, the feature will be visible.

The animation expression acts like a clock, generating cyclic progress values over a determinate period to match the values of its input. When a match occurs between the input and the progress value animation returns a 1 value and otherwise it returns a `0’.

I don't think this should be the first thing someone must read before using animation, but at the same time I don't think it has enough content to be it's own guide. I think that for this kind of case we should put some headings like "Advanced" or something like that that could be ignored by first-time users.

• Removing the "Introduction" part. Just by reading the previous paragraphs it's clear that some chapters are not an introduction but a combination of basic stuff, advanced stuff and some examples.

Regarding Ariana's comment I agree with her and I proposed the following solutions:

• In the Expressions guide we should talk more about how can they be combined. This is easy to illustreate with filter
• Using more examples. I think that here the main problem is the lack of embedded examples. As we talked other times, Mozilla CSS documentation is a really good comparison
• Improve the documentation about aggregation. In fact, I think this should be it's own guide.

[davidmanzanares]

What do you think about this index (still a stub)?

• Getting started

• Sources

  • Maps API (talk about agg and link to Maps API clustering)
    • Dataset
    • SQL (link to PgSQL docs?)
  • GeoJSON (minimal information and a example, talk about limitations and refer to Maps API source)
  • MVT (just mention and link to reference, mention this is advanced and unrecommended)

• Visualization

  • String and JS APIs
  • Viz properties
  • Expression types
  • Viz expressions (Introduction talking about nesting)
    • Numerical expressions (+ * / ^ log ...)
    • Color expressions (#FFF rgb rgba hsl hsv opacity)
    • Filter (showcasing nesting)
    • ramp (usages with colors and numbers) (talk about the amazing cielab interpolation)
      • legends
    • classification
    • Global aggregations
    • Viewport aggregations
    • Maps API clustering
    • Animation
    • Images (sprites), advanced (blend between images and operations like mul)
    • Ordering
    • Multi-scale cartography: zoomrange
    • placement

• Interactivity

  • Events
  • Variables
  • Link to viewport and global aggregations (special mention to ViewportFeatures)
  • Interactivity based styling

• Others

  • Text labels

[VictorVelarde]

Regarding to the different "sources", maybe they could be all included in a general category, like:

1. Data Sources
   1.1. CARTO: Maps API / Dataset & SQL
   1.2. GeoJSON
   1.3. MVT

In viz expressions, I would try grouping the long list into a fewer categories. Maybe something like:

• Basic: numerical, color, filter, ramp (and nesting)
• Medium: classification, aggregations, clustering
• Advanced: animation, sprites, multi-scale...

Not sure if the list is in complexity ascending order or it is just categorical 😄

[davidmanzanares]

Regarding to the different "sources", maybe they could be all included in a general category

Completely, I messed up the indentation, thank you! I fixed it

I wouldn't try to classify the difficultness of the expressions, because many times there is an easy usage and an advanced usage (animation is a good example of that). I think it's better to mark within each chapter which parts are advanced.

[makella]

So way back when we were prepping for Beta release at Locations NY, I think is when we just did a bunch of things fast to get some content out there... I totally agree with your first points about overall organization and flow of the guides!

Let's get them good and right and useful!

If you look at that past issue you'll see where I was going with organization but seems like we just didn't have the time at that time to make sure everything was all good (#362 (comment))

In addition to your comments, I also think that we need global organization of terminology, a standard way to follow the flow of examples, etc. Not only in the Guides, but throughout the Dev Center pages for VL.

This is my fault!! This was when I took @jgoizueta's animation guide and refactored to be less advanced and in that, looks like I made it too advanced. I think, like you say, we can really pear down the introduction and focus on the map-making process I tried to build through the guide.

Mixing advanced with basic stuff without notice. As an example read this (from the first parts of the Animation guide):
"The convention in CARTO VL is that 0 represents the boolean value false, (the absence of or an off state), while 1 represents true (the presence of or an on state). When the expression assigned to filter (animation in our case) has the value 0 for a given feature, that feature will be filtered-out and not be shown. If the value is 1, the feature will be visible.
The animation expression acts like a clock, generating cyclic progress values over a determinate period to match the values of its input. When a match occurs between the input and the progress value animation returns a 1 value and otherwise it returns a `0’.
"

Anyway, regarding your outline, I think it looks to cover most of the things we want. Can we put in the outline, the Guides that you think we can keep that we already have and/or the ones that you think need to be redone?

Also, we should probably sit down and talk about the logical flow of not only the Guides but also the examples and make sure that when we link to the Reference, that a description exists and that the language used is more consistent across everything.

To bring the beta Guides comment back, do you think it is worth going into the detail for each Guide, the purpose and learning lessons like I did for the initial ones?
(#362 (comment))

In addition to your outline, I would also add:

• Legends
• Point and Line Placement options (I think we are still working on this)

[davidmanzanares]

Yes, I totally agree, we were in a rush we did the best we could, which was more than fine for a Beta release IMO, but I think that now we should take it with calm and without distributing all the work between every person in the project to avoid fragmentation and increase cohesiveness.

a standard way to follow the flow of examples,

100% agree. Right now examples seem unrelated to the guides. We could use for them the same hierarchy we used for the guides and have links between each other, to allow users to swap between the real complete code and the explanation. Integrating the examples within the guides would be cool too.

Also, we should probably sit down and talk about the logical flow of not only the Guides but also the examples and make sure that when we link to the Reference, that a description exists and that the language used is more consistent across everything.

Yes, and I think we should do this in both ways. We should probably add links in the reference to our guides. If someone is looking at the animation reference, it should just take a click to go to the animation guide.

Can we put in the outline, the Guides that you think we can keep that we already have and/or the ones that you think need to be redone?

I think this is difficult to answer because most of the current content is dispersed in more than one guide and there is no 1 to 1 relationship between my proposed outline and the previous outline. In any case, I think we already have most of the content. Probably the first step should be to refactor the guides without completing the things that weren't done previously, but moving the content that we already had.

To bring the beta Guides comment back, do you think it is worth going into the detail for each Guide, the purpose and learning lessons like I did for the initial ones?

What do you mean?

In addition to your outline, I would also add:

Totally, I forgot about those. Regarding placement, I think everything planned for the public release is already implemented. I'm going to update my previous comment.

[makella]

but I think that now we should take it with calm and without distributing all the work between every person in the project to avoid fragmentation and increase cohesiveness.

YES!

What do you mean?

Basically, should we stub out an outline for each Guide and build on the concepts in a logical order? If you look at the beta outline, you'll see that was the idea... "Now that we've introduced this, we can introduce this":

[rochoa]

Some thoughts after reviewing the existing guides with @davidmanzanares:

• Getting started: pretty much OK, the example should be embedded.
• The basics of the syntax: touches too many things. For instance, variables, which might be better introduced when talking about interactivity. Mixing JS and String API leads to confusion: too many options.
• Introduction to expressions: although is a good idea, it starts explaining eval, which is too advanced. Things like color: red + blue or talking already about aggregations. It looks more like the reference for expressions.
• Introduction to interpolation: it should go after "Introduction to style" to start with. Probably we should re-use this content for other guides.
• Introduction to styling: this looks more like the reference for styling, talking about things that are not 100% related to style: filters and aggregations, e.g. resolution: ....
• Introduction to animation: we need to focus more on the basics and later introduce complex concepts, e.g. filter 0-1 values convention.
• Introduction to interactivity: this looks great, it's good for introducing variables.
• Introduction to image files: naming is a bit weird. Symbol vs Images.

General comments:

• Examples should be embedded always.
• Every guide states is an "Introduction". Redundant?

[rochoa]

As a summary, I think the proposed outline by @davidmanzanares with some of the touches proposed by @VictorVelarde, we could have a better structure. Instead of having a lot of separate guides, having more in deep ones focused on big topics where you learn more about the topic as you progress through the guide. At least for a first public release that has more value.

We could work on specific guides if we find some people struggle with some advanced topics. Another alternative is to save those "guides" for writing some blog posts or a nice examples section.

[makella]

@rochoa building off what you said,

Instead of having a lot of separate guides, having more in deep ones focused on big topics where you learn more about the topic as you progress through the guide

I also think that some of the topics are covered in the "Examples" section and many times, people like to find an example, copy the code and then start customizing so I definitely think we should have a good balance between the two and after taking a closer look at the proposed outline, it seems like some of the topics can go straight to examples. In many cases we already have examples that we can likely reuse.

In order to make the link more direct between guides and examples, at the end of EACH guide, we should put links to other resources that are similar or good background information for the topic of the guide.

@davidmanzanares I think this also goes to your point of some of the "Introduction" paragraphs in the Guides being too advanced. This is the pitfall (I feel like) of writing a Guide on a given topic... there is the inclination to explain the why before going into the how.

For example, looking through the Viz section, my instinct is to break this down into what would be a valuable GUIDE (G) vs a valuable EXAMPLE (E)

• Viz expressions (Introduction talking about nesting): G
• Numerical expressions (+ * / ^ log ...) G
• Color expressions (#FFF rgb rgba hsl hsv opacity) E
• Filter (showcasing nesting) E
• ramp (usages with colors and numbers) (talk about the amazing cielab interpolation) G
  • legends
• classification E
• Maps API clustering G
• Animation G
• Images (sprites), advanced (blend between images and operations like mul) G
• Ordering E
• Multi-scale cartography: zoomrange G
• Placement E

I think these two could be combined into one: G

• Global aggregations
• Viewport aggregations

[makella]

I wanted to open a different issue to continue this conversation since the title is only partially relevant at this point! (haha)

As we've been discussing, there is a need to rethink/restructure the content we have in the Dev Center for public release so figured it a good idea to start organizing things in a spreadsheet so we can keep track/all see:

https://docs.google.com/spreadsheets/d/1M5sKwIkkAD_DOp2zbM1WhST3waHN2BY2ICdcW4J-ztA/edit#gid=0

In the GUIDES tab,

• I added all the EXISTING guides with all of the comments that @rochoa put in his comment here the other day
• I also added a section for PROPOSED where I put in everything from @davidmanzanares comment here

EXAMPLES tab,

• EXISTING examples, with comments so we can track those easily too

Hope this is a good start for us.

It would also be nice to have all the reference terminology in a spreadsheet or something so we can review and check for inconsistencies.

If we have a reference dictionary type thing that we could all use when writing about the technical pieces, that might help unify the overall voice of the Dev Center.