WASM & optimization

[Caellian]

Have you considered using Rust & WASM for (parts of) generator? This comes from a completely selfish want of mine to use such modules as part of a larger project of mine, but I'm just asking.

[Azgaar]

WASM - yes, but it didn't find a good place. JS code is pretty fast, my Mac generates a map of default settings in 0.3s. That's with all calculations we have. Graphics is slow, but it's SVG so there is no room for improvement here.

[Caellian]

Oh right. That makes sense.

I really like the idea of showing actual export format in interactive form, but for some reason interactive SVGs are the most unoptimized part of browser rendering especially so on WebKit (Safari).

[goteguru]

I had the same idea some time ago, but quickly realized SVG is the bottleneck, the Javascript generator pretty much optimized and fast with those typed arrays. I've experimented with SVG pixel cache (ie. cache svg vector data in hierarchic pixel buffers for view and use SVG for editing only) but I failed to make it smooth and in sync.

All fast map services (google maps, leaflet, etc) use this hybrid method: (WMS based) raster accelerated background + thin vector overlay. This feature might be a good place for using WASM + rust. With new browser features like OPFS the pixel cache won't even hog the memory anymore and can be a viable solution. If someone smarter than me (or at least having more time) can do it, that may have a vast performance improvement (Or maybe not. :) Anyways, if you are interested I'm inclined to run an another round, I always wanted to use some rust.

[Caellian]

That is something that comes to mind and my guess is that just moving the background components to a static layer would see a large improvement because all those SVG paths likely take up majority of render time in the browser.
When I enable all features, ~50% of draw call goes to ViewportFrame::BuildDisplayList and the other half to actually drawing the contents on Firefox.

[goteguru]

Yes, but probably the SVG engine implementation already does some caching, not that extensively as we need of course. Otherwise I agree, it might have huge effect, but not easy to do properly. SVG layers can change, we must implement several transparent pixel layers or invalidate the whole cache on layer changes. Moreover almost everything can be edited, therefore the node and all the cache up the quadtree (or similar data structure) must be invalidated. For interactive actions we must switch back to SVG (most of the code is very tightly coupled with it) therefore the pixel cache should always be in sync with the SVG representation...

I think the best way to do this would be using service workers (cache update won't hurt user experience) but I'm not sure if it is viable or not.

[Azgaar]

Agree. SVG engines are ancient and, as fas as I know, no one is working on them. I was hoping Chrome or other team will do the work and give us gpu-optimized engine, but now I'm pretty sure they are not interested in it. SVG is a small niche and what we are doing with it is pretty much extreme.

So we basically have 3 options:

• leave it as is and hope for the best. SVG is really good format to work with. It's easy to write and really easy to debug
• rasterize some or all layers. This will improve the performance a lot, but will make it harder to be interactive. Probably we can restrict zoom to 5 positions only so that we don't need to rerender raster layers too frequently.
• render map in 3D. For historical reasons 3D engines, even in browser, are very fast. 3D doesn't mean it should look 3D, it can be styled to pretty much anything, e.g. see this old Amit's project: redblobgames.com/maps/mapgen4. The problem with 3D rendering is that is has complex API and we are loosing all benefits of svg.

[Azgaar]

Yes, probably. It takes less that 1 second for me.

[goteguru]

I have a huge ultrawide display. Probably that. On FHD it's much faster. 1 sec is still way too long. Zooming and scrolling in the SVG is faster than that probably because it don't have to re-build the svg structure. I wonder if it is possible to capture the pixel content of the SVG element somehow.

[Azgaar]

1 sec is for download, general regeneration and rendering takes 200-300 ms for me. For SVG you cannot get pixel content as it doesn't have it, or at least it doesn't expose it.

[goteguru]

1 sec is for download, general regeneration and rendering takes 200-300 ms for me.

Oh you mean the render time. I measured the "png export" time because we need the pixel content for the cache. The image conversion probably adds considerable overhead. (For some reasons jpg export is faster, probably because it's smaller).

For SVG you cannot get pixel content as it doesn't have it, or at least it doesn't expose it.

Yes , I didn't find any interface to that (while it certainly has a rasterizer, and it's caching the image, pixels are clearly visible while zooming. But that's won't help if we cannot extract it fast enough.

Ideally, we should render the svg on OffScreenCanvas in webworker, create the cache files and use WMS-like client (or an actual WMS client) to display it to the user. This way we could even utilize multiple cores. Unfortunately the support is sparse but may improve over time.

Switching between edit / view modes would require to re-render (recreate) the svg object though...

[Caellian]

Switching between edit / view modes would require to re-render (recreate) the svg object though...

Not fully - you can still get benefits of rasterized content if you keep things you aren't editing static. E.g. you click on edit rivers - keep everything else as tiles and only show rivers as editable svg; when the editing changes, rasterize rivers and show SVG of whatever is picked next.