Map cluster only 'shows' 5,000 of 11,000 markers?

jlayt · May 17, 2024, 12:55pm

Hi,

I have a site with 11,669 items with map markers published to it (per the advanced search), but the map cluster when zoomed out only shows a count of 5000 markers. Zooming in shows large areas with no markers where expected. Is there a setting somewhere I’m missing that imposes a maximum?

John.

fackrellj · May 17, 2024, 2:16pm

Hi @jlayt ,

I just did a quick look at the code and it does look like there is hard-coded 5000-item limit.

github.com

omeka-s-modules/Mapping/blob/abb87e0381f26988c105898d122c8276aea5d1c3/src/Controller/Site/IndexController.php#L24


      
              // Respect the browse_attached_items setting.
              $itemsQuery['site_attachments_only'] = true;
          }
          
          // Only get items that are in this site's item pool.
          $itemsQuery['site_id'] = $this->currentSite()->id();
          // Only get items that have features.
          $itemsQuery['has_features'] = true;
          // Limit to a reasonable amount of items that have features to avoid
          // reaching the server memory limit and to improve client performance.
          $itemsQuery['limit'] = 5000;
          // Do not include geographic location query when searching items.
          unset(
              $itemsQuery['mapping_address'],
              $itemsQuery['mapping_radius'],
              $itemsQuery['mapping_radius_unit'],
          );
          $itemIds = $this->api()->search('items', $itemsQuery, ['returnScalar' => 'id'])->getContent();
          
          // Get all features for all items that match the query, if any.
          $features = [];

jlayt · May 21, 2024, 10:11am

Thanks, should have grepped for that myself… Looks like I’ll have to fork the Mapping code to override that and test what impact it has, then submit a patch to allow it to be set in the config or admin panel.

On our current site we use OpenLayers with WebGL and even with 12,000 points and no clustering it’s very smooth, but then we only load the points, the pop-up info gets populated via API call. Will have to see if Leaflet can be similarly configured.

Good thing I costed an extra week to get mapping working the way we want!

jlayt · May 22, 2024, 5:11pm

Hmmm, might look at using GitHub - mapbox/supercluster: A very fast geospatial point clustering library for browsers and Node. which claims to be able to cluster millions of points in a fraction of the time.

John.

jimsafley · May 22, 2024, 6:34pm

Keep in mind that the limit is on the number of items, not the number of markers. Since one item can have many markers, the upper limit of what’s rendered on the map can be much higher than 5000.

We’re going to implement pagination for map browse in a future release. That way users can paginate through the entire result set, 5000 items at a time. I’m aware of the supercluster library, but I prefer to use the clustering plugin that is maintained by Leaflet.

jlayt · May 24, 2024, 2:48pm

Thanks Jim. Looking forward to the pagination support, although it’s possibly not what our client is after for this site.

Would you be open to a patch to allow the 5000 to be overridden in the local.config file but not in the admin interface, so those of us who need to can, rather than having to change the source after each update?

Also, I note that you’ve rewritten Mapping to support generic geometries not just points. Do you have a rough idea of when that will be released, as the site we’re developing would benefit from switching to that now before it goes live in 2-3 months. Happy to be a tester for it before then.

Thanks!

John.

jimsafley · May 24, 2024, 3:56pm

Would you be open to a patch to allow the 5000 to be overridden in the local.config file but not in the admin interface, so those of us who need to can, rather than having to change the source after each update?

A configurable limit is a good idea, pagination or not. No need for a PR. I’ll go ahead and implement it as a site setting myself.

Also, I note that you’ve rewritten Mapping to support generic geometries not just points. Do you have a rough idea of when that will be released, as the site we’re developing would benefit from switching to that now before it goes live in 2-3 months.

We’ve just finished the testing and documentation phase and are waiting on a few details to be worked out. It’ll certainly be released within the next two weeks. Feel free to try it out yourself, if you would like: GitHub - omeka-s-modules/Mapping