rundel / carto-parser

Hi there 👋

carto-parser's Issues

architecture document

Page doc on how boost spirit works and how the parser works at a high level.

Basically, what should developers know before getting involved.

Linking failure with gcc 4.6.1 on Ubuntu 11.10

@rfw New scons build system looks good but I'm getting a linking error when using g++ 4.6.1 that I don't see with clang++ 3.0.

Mapnik trunk is built using clang but it seems like gcc is having trouble seeing the boost libraries as well.

https://gist.github.com/1756439 shows the error output for "CXX=g++ make"

"CXX=clang++ make" builds and links cleanly.

Zoom level filters

Zoom levels are treated like any other filter and do not create Max and Min scale attributes.

README pls

Would be great to have a summary readme with overview, how to build, etc.

LInux (Gentoo) make fails: expr_grammar has incomplete type

themis carto-parser # make clean
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Cleaning targets ...
scons: done cleaning targets.
themis carto-parser # make
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
cxx: src/mml_parser.cpp -> src/mml_parser.o

In file included from src/mml_parser.cpp:9:0:
include/mss_parser.hpp:34:61: error: field 'expr_grammar' has incomplete type
compilation terminated due to -Wfatal-errors.
scons: *** [src/mml_parser.o] Error 1
scons: building terminated because of errors.
make: *** [all] Error 2
themis carto-parser #

Error: Failed to parse expression: ""

I get that, and no line number output, when trying to parse https://github.com/mapbox/foss4g-benchmark-style

parsing mss referenced by mml

right now the example just exposes parsing one or the other which makes sense for testing, but we need an easy way to parse both at the same time. Perhaps its time to make that the default behavior?

open-streets-dc won't parse

d:carto-parser dane$ ./carto-parser ~/Documents/MapBox/project/open-streets-dc/style.mss 
Error: Parser failed!

./carto-parser ~/Documents/MapBox/project/open-streets-dc/labels.mss 
<?xml version="1.0" encoding="utf-8"?>
<Map srs="+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs"/>

d:carto-parser dane$ ./carto-parser ~/Documents/MapBox/project/open-streets-dc/highway.mss 
Error: Parser failed!

Cannot perform multiplications with color and scalar value

The following fails:

color: #fff * 0.7;

Since #fff gets turned into a utree node, multiplication by 0.7 is not possible in carto::expression::eval_node as it directly tries to multiply a utree with a double.

The only way I can see around this is to create a new data structure for colors that overloads arithmetic operations, but it may require an overhaul of internals.

shared_ptr attributes and inheritance

Currently inheritance of rules is handled using copy constructors:

mapnik::rule rule(parent_rule);

this is problematic for shared_ptr attributes like placement_options_ in text_symbolizer as all the subrules end up sharing the same shared_ptr. Not sure what the best option would be, my thinking is to come up with some way to copy these pointers with some switch in the copy constructor.

Thoughts @springmeyer?

better error reporting

./example test.mml (when the shapefile is missing) gives:

d:carto-parser dane$ ./example test.mml 
Error: Unknown exception at Line: 13 Col: 0
<?xml version="1.0" encoding="utf-8"?>
<Map srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs"/>

I would expect the error being on line 9, and explain the reason.

I assume you have been focusing first on error reporting in .mss first, which is the more important, but wanted to flag this since I saw it.

use filter-mode="first"

carto.js sets this as per http://trac.mapnik.org/ticket/706 for every style to avoid having to negate filters.

missing parsing of mss within mml

running ./example test.mml gives the below xml output. Looks perfect except that the 'style' is missing.

<?xml version="1.0" encoding="utf-8"?>
<Map srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs">
    <Layer name="world" srs="+proj=latlong +ellps=WGS84 +datum=WGS84 +no_defs">
        <Datasource>
            <Parameter name="file">./data/world_borders</Parameter>
            <Parameter name="type">shape</Parameter>
        </Datasource>
    </Layer>
</Map>

arrays of errors

carto.js tries to return an object that contains an array of errors so that the tilemill interface can highlight various parts of the css syntax that need attention by the stylesheet author.

I'm not aware of how much or how hard this would be to match in carto c++, so just wanted to flag it for discussion. Obviously right now the most useful thing is graceful error printing and compiling as much as possible down to XML, so we can quickly gut check the conversion.

But, hopefully in not too long we can start wrapping this in api functions that will expect an object of some sort returned with all the errors (and also perhaps all the more minor warnings) in a way that a UI could do some fantastic/flexible things with.

document minimum boost version

is it 1.45, 1.46? Any idea? Clang could not compile boost before 1.45 I think, but gcc could, so keep in mind if you remember what gcc version works.

sample data - pls check it in

would be great to have the data used in the .mml files (with relative paths) checked in so we don't have to go hunting to test.

migration to mapnik core

As soon as we can we should:

1. fork or branch https://github.com/mapnik/mapnik/ and start integrating this code

2. create a new label for "css" at https://github.com/mapnik/mapnik/issues

3. move remaining issues over to https://github.com/mapnik/mapnik/issues and label them

4. assign to the mapnik 2.1.x milestone (or 3.x depending on when @rundel and @rcoup feel like stuff will be ready to merge)

mml errors are fatal mss errors are not

This:

diff --git a/tests/map.mss b/tests/map.mss
index 8ac59f1..5b79b5a 100644
--- a/tests/map.mss
+++ b/tests/map.mss
@@ -90,7 +90,7 @@ Map {
 #populated_places {
     .popplaces {
         {
-            text-name: "[GEONAME]";
+            text-name: "[GEONAME";
             text-face-name: "DejaVu Sans Book";
             text-size: 10;
             text-halo-fill: rgb(255, 255, 200);

throws the error but also outputs XML. Should it be that way?

d:carto-parser dane$ ./carto-parser tests/map.mss
Error: Failed to parse expression: "[GEONAME" at Line: 101 Col: 1
<?xml version="1.0" encoding="utf-8"?>
<Map srs="+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs" background-color="rgb(255,255,255)">
    <Style name="ontario_hydrography">
        <Rule>
            <Filter>([HYC]=8)</Filter>
            <PolygonSymbolizer fill="rgb(153,204,255)"/>
        </Rule>
    </Style>
    <Style name="provinces">
        <Rule>
            <Filter>([NAME_EN]=&apos;Ontario&apos;)</Filter>
            <PolygonSymbolizer fill="rgb(250,190,183)"/>
        </Rule>
        <Rule>
            <Filter>([NOM_FR]=&apos;Québec&apos;)</Filter>
            <PolygonSymbolizer fill="rgb(217,235,203)"/>
        </Rule>
    </Style>
    <Style name="provincial_borders">
        <Rule>
            <LineSymbolizer/>
        </Rule>
    </Style>
    <Style name="quebec_hydrography">
        <Rule>
            <Filter>([HYC]=8)</Filter>
            <PolygonSymbolizer fill="rgb(153,204,255)"/>
        </Rule>
    </Style>
    <Style name="roads">
        <Rule>
            <Filter>([CLASS]=1)</Filter>
            <LineSymbolizer stroke="rgb(188,149,28)" stroke-width="7" stroke-linejoin="round" stroke-linecap="round"/>
        </Rule>
        <Rule>
            <Filter>([CLASS]=1)</Filter>
            <LineSymbolizer stroke="rgb(242,191,36)" stroke-width="5" stroke-linejoin="round" stroke-linecap="round"/>
        </Rule>
        <Rule>
            <Filter>([CLASS]=2)</Filter>
            <LineSymbolizer stroke="rgb(171,158,137)" stroke-width="4" stroke-linejoin="round" stroke-linecap="round"/>
        </Rule>
        <Rule>
            <Filter>([CLASS]=2)</Filter>
            <LineSymbolizer stroke="rgb(255,250,115)" stroke-width="2" stroke-linejoin="round" stroke-linecap="round"/>
        </Rule>
        <Rule>
            <Filter>(([CLASS]=3) or ([CLASS]=4))</Filter>
            <LineSymbolizer stroke="rgb(171,158,137)" stroke-width="2" stroke-linejoin="round" stroke-linecap="round"/>
        </Rule>
    </Style>
</Map>

support attachments

As described in the readme: https://github.com/mapbox/carto/blob/master/README.md attachments are syntax invented to allow you to predictably apply multiple styles to a given layer. Sometimes in mapnik this is referred to as aggregate styles, and is an important syntax to support road casings:

(from link)

Attachments can be nested and they can be named whatever you choose. The key thing is that when one is encountered the styles within it are pushed into another mapnik.Style and attached to the layer's style array in the order encountered.

So carto like:

#countries {
  ::outline {
    line-color: #85c5d3;
    line-width: 2;
    line-join: round;
  }
  polygon-fill: #fff;
}

// or 

#countries::outline {
    line-color: #85c5d3;
    line-width: 2;
    line-join: round;
}
#countries::fill {
  polygon-fill: #fff;
}

should both end up like:

<Map >
<Style name="countries-fill" filter-mode="first">
  <Rule>
    <PolygonSymbolizer fill="#ffffff" />
  </Rule>
</Style>
<Style name="countries-outline" filter-mode="first">
  <Rule>
    <LineSymbolizer stroke="#85c5d3" stroke-width="2" stroke-linejoin="round" />
  </Rule>
</Style>
<Layer name="countries">
    <StyleName>countries-outline</StyleName>
    <StyleName>countries-fill</StyleName>
    <Datasource> ... </Datasource>
  </Layer>
</Map>

Don't throw if unknown properties are encountered in .mml

The MML syntax is evolving - basically as more map/render level properties are needed. Currently the carto parser errors out on unknown properties - ideally each of these properties could be collected and stored for later use (for now, after collecting them, they could be output in a warning after parsing).

For context see discussion at mapbox/tilelive-mapnik#4

Filter inheritance

Nested styles are not inheriting their parents filter, this appears to happen because to_expression_string returns "true" even when the filter exists. See parse_filter in mss_parser.hpp.