Feature/mvt integration

[rkreienbuehl]

@p-lr here the new MR draft of my work.

Rasterizer is still very slow on all devices (desktop seemed fast at first, but depends on the hardware), performance needs to be drastically improved, maybe you have some ideas what could bring better performance? One idea is to draw the symbols instead of using markers, but rendering itself also needs to be improved. I will investigate this further. Maybe you also have some inputs on how to improve performance?
I could fix the crashes on iOS and Android, so the demo works on iOS, Android and desktop
Collision detection is not integrated right now
I am in holidays from next week, but will work on it as I find the time.

[rkreienbuehl]

@p-lr I improved the performance quite a bit with some LruCaches. It is still not perfect, but I noticed a big difference (when testing with Symbol markers commented out). Displaying symbols as markers is not ideal, maybe drawing them in a separate layer would improve performance.

[p-lr]

I will have a look this weekend. If I remember correctly, markers are used for symbol rendering because symbols can span across tiles, and markers is currently the only way to position some "view" at a known position. However it requires careful management because too many markers can impact performance.
I'll let you know.

[rkreienbuehl]

That's what I thought, but I didn't had the time to look more into it. Curious to hear from you.

[p-lr]

In addition, when you rotate the map, symbols like place or street names must remain straight. So you see that they are handled as markers. Only we need to figure out a way to do it efficiently.

[rkreienbuehl]

Thats true. I am on it to bring symbols into a separate layer, so no markers need to be added. It works on desktop, on iOS there is a problem, will work on it more in the evening and push it as soon as there is a first milestone on it.

[p-lr]

I guess the symbols layer mostly replicates how the marker layout works. But yes, it's a good idea to have a core concept dedicated to symbols, instead on building around the markers api.

[rkreienbuehl]

Yes, instead of render() on a symbol, i implemented a draw() method that takes the drawScope and the draws on it mostly the same way how render() worked.

[rkreienbuehl]

Symbol painting is now in a separate layer. This works, but it is not tested into detail.
Right now, the rendering performance is the biggest problem, therefore symbols are painted delayed.

[p-lr]

So I finally finished some work on another app and I have more time for MapCompose now.
Did a quick test, and the new "Vector tile demo" works on android -- good job!
As you said, there's work to do on symbols. I zoomed on Ajaccio (the capital of Corsica) and this what loaded after ~2mins ><:

No doubt there's a lot of work on my side to review the massive amount of new code.
Instead of submitting a huge list of remarks, I prefer to go step by step. The first thing I noticed is the code added to MapState:

private val mViewportInfoFlow = MutableStateFlow<ViewportInfo?>(null)
    val viewportInfoFlow: StateFlow<ViewportInfo?>
        get() = mViewportInfoFlow

Analyzing further, it seems to be used by mapLibreDebugApp, which was the experimental demo. If I understand correctly, your newly added demo supersedes this right? So we can delete this mapLibreDebugApp? If you confirm, I can remove it and refactor this part (as I believe the viewportInfoFlow could be exposed in a cleaner way) if you allow me to push on your branch.

Thanks again for your work!

[p-lr]

Alternatively, we can merge your MR right now, so I can contribute on feature/mvt-integration directly.

[rkreienbuehl]

I just added you as collaborator.

Analyzing further, it seems to be used by mapLibreDebugApp, which was the experimental demo. If I understand correctly, your newly added demo supersedes this right? So we can delete this mapLibreDebugApp? If you confirm, I can remove it and refactor this part (as I believe the viewportInfoFlow could be exposed in a cleaner way) if you allow me to push on your branch.

Thats correct. I didn't do any cleanup so far and left the mapLibreDebugApp module as is, if I would have needed something to lookup when working on it, bit this module will not build anymore.

[p-lr]

I've removed the former demo, and refactored core classes like MapState and TileCanvasState.
Now, the internal concept of ViewportInfo is generated in the body of MapState.addVectorLayer where this flow is collected. I left a comment to refactor this later because it adds clutter in this function.

My next focus will be the high level architecture of MapState.addVectorLayer. Like for example, getting a Rasterizer is done using a suspending function. The reason is the style needs to be fetched. Once we get a rasterizer we update a var in the body, as we need the rasterizer for symbols. This tells that we cannot start producing symbols until the rasterizer is ready. So my plan is to refactor this part to get a better high level view.

[p-lr]

After a quick analysis. I could spot two causes for symbols performance issue.

1. Way too many symbols are drawn. This hogs the main thread. See the screenshot below. A total of ~5k symbols. We should expect a few hundreds at most.
2. Symbols are produced in such a way that tasks accumulate. In the example below, it took several seconds to generate the 5k symbols. But if I zoom in then out, several symbol production tasks will pile up and the delays accumulate.

To fix issue 1, we'll have to limit the number of produced symbols. By using collision de detection and/or using some algorithm, and also maybe using a hard limit.
To fix issue 2, I will use a similar technique as in TileCollector.

But before going through those fixes, like I said in my previous comment, I need to refactor the MapState.addVectorLayer function.

Don't hesitate to let me know that you read my messages.

[rkreienbuehl]

Yeah, I noticed the same. At the moment every symbol is drawn. I will research how maplibre handles symbols and implement such a behavior to address issue 1.

[p-lr]

I will let you know when I'm done with the refactor. I'll see if I can also tackle issue 2 at the same time.
I also noticed that tiles rendering can be optimized but that will require some major changes to the core library, so it isn't the priority.

[rkreienbuehl]

Sounds good to me. I'll find out more about collision detection and involved algorithms in the meantime.

[p-lr]

I've partially improved issue 2, or at least enough for the moment. Two major changes:

• Upon viewport change, any ongoing symbol processing is cancelled. This is done with:

viewportInfoFlow
    .throttle(250)
    .collectLatest { viewportInfo ->
...
}

• Switched to processing symbols using Dispatchers.Default

I believe this is enough for issue 2 (for now). You can focus on issue 1.

[rkreienbuehl]

I made some research. What I found out so far:

To come closer to how maplibre does symbol placement, symbols should be preproduced so I will implement a placement engine. This would improve performance, because symbols aren't produced every time the viewport changes. Also with the current implementation of symbols only the priority is used for determining the order of placing them, maplibre also uses the layer as info for this.
When the viewport changes, I need only to place symbols (and do collision detection) instead of rendering them every time.

Will take some time to implement this properly. Will get back as soon as there is an update on it.

[p-lr]

Having a SymbolsEngine feels necessary indeed. It might be interesting to discuss the design before starting the implementation.
On top of my head, there are two different things that needs to be clarified. Currently, our "viewport" is a combination of two very different data:

• the tile set
• the visible area

On top of my head, the engine should make this distinction. Like, when the tile set changes, that means new tiles are visible (or a completely different list of tiles). This is different from when the visible area changes (alone), as in this case it means the user has zoomed in or out but the tiles set is still the same. In that case, the internal list of symbols remains the same, only the scale changes and the engine should probably run collision detection on all the markers in cache to determine the new list of visible symbols.

To summarize, the engine would maintain an in-memory cache of produced symbols, and that cache would be updated whenever the tile set changes. Maybe that cache would look like a Map<Tile, List<Symbol>>, to that symbols are cached per tile (e.g. some sort of 2D cache). That cache would contain several thousand symbols in total, as we've seen. However, thanks to collision detection and the logic you mentioned, only a limited list would be sent to the ui for rendering.

What do you think?

[rkreienbuehl]

I totally agree. This approach would also be very close to what maplibre does with symbols. I didn't had the time to come up with a full concept, but will work on it this week as much as I find time, so we can discuss this in detail.

[p-lr]

Sounds good, no hurry.