Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving MapStore Performances #4675

Open
5 tasks
tdipisa opened this issue Dec 19, 2019 · 8 comments · Fixed by #10459 · May be fixed by #10442
Open
5 tasks

Improving MapStore Performances #4675

tdipisa opened this issue Dec 19, 2019 · 8 comments · Fixed by #10459 · May be fixed by #10442

Comments

@tdipisa
Copy link
Member

tdipisa commented Dec 19, 2019

Description

MapStore need to be improved in order to optimize some loading performances and functional behaviors.
We should take into account PageSpeed Insights hints to optimize performance and verify at the end the effective improvement. The list below comes from a preliminary analysis of PageSpeed Insights and this Epic collects the main involved points of interest:

  • MS bundle: the bundle of MS need to be reduced, the issue Investigated on bundle sizes( too high) and memory usage during build #4297 provides a recent reports of currently used libraries. Action here are:

    • ogc-schemas: GetCapabilites parsing error for inspire_vs:ExtendedCapabilities #6489 is the biggest library (4.21 MB (11.8%)) and it is currently used mainly for some specific purposes like CSW requests and parsing of CSW results. It would be useful to remove this dependency by replacing it with a proprietary solution in MS (estimate for this are ~5/8 pts). This library, together with pdfjs and shpjs anyway is loaded dynamically, therefore should not affect the initial load of the bundle. We have to verify if it is imported somewhere else in the code.

    • Review of dependencies: the whole list of dependencies need to be analized to check if something can be excluded (clean up of unused/old dependencies).

  • Mapping Libraries: by default MS uses Leaflet for mobile and OL for desktop (Cesium is an option). Mapping libraries should be loaded only when needed depending on the used device. This means that we can avoid loading OL in Mobile devices and Leaflet in Desktop if not specifically configured. At the same time Cesium JS should not be loaded if the plugin is not enabled in configuration. This point also involves the MS bundle, because Investigated on bundle sizes( too high) and memory usage during build #4297 reports also part of the bundle that currently include some OL and Leaflet scripts (is the report considering dynamic dependencies?). We should split the bundle also per mapping libraries if needed.

  • Other dependencies loading: a lazy loading of scripts should be implemented to load only needed dependencies related to the application context in use (home/viewer/dashboard/stories). This means to properly split the bundle according to required functional behaviors. For example I don't need to load viewer scripts if I'm loading the home page.

  • Home Page Loading: the loading of the MS's landing page can be improved also in other terms. Currently all first pages of resource categories (Maps, Dashboards, Stories) are loaded at the same time: by default only the list of Maps should be retrieved from the backend and the other should be loaded only if visualized.

  • CDN: at the release we can evaluate to speed-up script loading by deploying and getting the new release bundles from a CDN.

What kind of improvement you want to add? (check one with "x", remove the others)

  • Minor changes to existing features
  • Code style update (formatting, local variables)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • CI related changes
  • Other... Please describe:

Other useful information

Consider that MS already implements some lazy loading of scripts; therefore existing mechanisms, currently used for specific piece of the framework, can be extend to the entire application.
For each of the above point we can create a separate issue to connect to this epic.

Tasks

  1. Epic enhancement
    allyoucanmap tdipisa
  2. Internal investigation
    tdipisa
  3. StyleEditor
  4. Good first issue enhancement
    tdipisa
  5. StyleEditor bug estimate needed investigation
    allyoucanmap
@mbarto
Copy link
Contributor

mbarto commented Feb 4, 2020

A quick report on reducing initial js loading. Playing with dynamic splitting of pages and plugins this is what we can get.

This is the current loading of js files for the homepage.

before

This is the after code splitting.
after

Instead of 2.5Mb we download about 782kb.

Moreover the code that is not red boxed can be removed from initial loading, because it's not needed by the homepage. So 782kb could be the complete payload for js code.

The experiment code is here: https://github.com/mbarto/MapStore2/tree/split_pages

@mbarto
Copy link
Contributor

mbarto commented Feb 4, 2020

Some stats about home page loading payloads:

Type of Resource Size
JS 3590Kb
IMG 1300Kb
CSS 125Kb
Font 63Kb
XHR 44Kb
HTML 21Kb

@tdipisa
Copy link
Member Author

tdipisa commented Feb 4, 2020

Thank you @mbarto, let's discuss this in a meeting.

@tdipisa
Copy link
Member Author

tdipisa commented Feb 13, 2020

As a first step, we agreed to move on with the following:

  • Mapping Libraries
  • Other dependencies loading
  • Home Page Loading

@srtonz
Copy link
Contributor

srtonz commented Mar 26, 2020

Just subscribing here as we're very interested in seeing improvements on this issue and #4297 - let me know if there's anything you need help with

@offtherailz
Copy link
Member

offtherailz commented Apr 9, 2020

Added an improvement to increase general rendering performances, see #5133
This should improve general responsiveness of the application on tile loading and when, in general, many redux actions are emitted frequently.

@allyoucanmap
Copy link
Contributor

allyoucanmap commented Sep 1, 2022

@tdipisa Updete 01 September 2022

MS bundle: the bundle of MS need to be reduced, the issue #4297 provides a recent reports of currently used libraries. Action here are:

ogc-schemas: #6489 is the biggest library (4.21 MB (11.8%)) and it is currently used mainly for some specific purposes like CSW requests and parsing of CSW results. It would be useful to remove this dependency by replacing it with a proprietary solution in MS (estimate for this are ~5/8 pts). This library, together with pdfjs and shpjs anyway is loaded dynamically, therefore should not affect the initial load of the bundle. We have to verify if it is imported somewhere else in the code.

Review of dependencies: the whole list of dependencies need to be analized to check if something can be excluded (clean up of unused/old dependencies).

ogc-schemas is still inside the source code but is not imported in the main bundle, we should remove these library anyway if possible. We analyzed some dependencies and removed from the main bundle the one related to specific components, these library will be loaded only when the related component is loaded (see #8535).

Mapping Libraries: by default MS uses Leaflet for mobile and OL for desktop (Cesium is an option). Mapping libraries should be loaded only when needed depending on the used device. This means that we can avoid loading OL in Mobile devices and Leaflet in Desktop if not specifically configured. At the same time Cesium JS should not be loaded if the plugin is not enabled in configuration. This point also involves the MS bundle, because #4297 reports also part of the bundle that currently include some OL and Leaflet scripts (is the report considering dynamic dependencies?). We should split the bundle also per mapping libraries if needed.

We improved the mapping libraries loading so now they should be loaded once the map type match the name of the library and removed also the leaflet cdn in the html head.

There are still some issue to be investigated:

  • OpenLayers is included in the main bundle anyway, probably some classes are used in the utils
  • we are importing all the components related to map supports as soon as the map type is selected, we should instead verify if some of these supports are importing additional dependencies and in that case opt for a dynamic loading

Other dependencies loading: a lazy loading of scripts should be implemented to load only needed dependencies related to the application context in use (home/viewer/dashboard/stories). This means to properly split the bundle according to required functional behaviors. For example I don't need to load viewer scripts if I'm loading the home page.

The modularization of plugins introduced the lazy loading of plugins (see #8423). We have still a list of static plugins because of issue related of the app lifecycle, these are the plugin still static that could be additionally be transformed in dynamic:

ContextCreatorPlugin,
ContextPlugin,
Dashboard,
DashboardsPlugin,
FeedbackMaskPlugin,
GeoStoryPlugin,
GeoStoriesPlugin,
MapsPlugin,
PrintPlugin,
RulesDataGridPlugin,
RulesEditorPlugin,
RulesManagerFooter,
LoginPlugin

We did some tests but at this time the modularization of these plugins was not improving much the initial loading (proof of concept here #8501). Anyway we could take this into account for future improvements.

Home Page Loading: the loading of the MS's landing page can be improved also in other terms. Currently all first pages of resource categories (Maps, Dashboards, Stories) are loaded at the same time: by default only the list of Maps should be retrieved from the backend and the other should be loaded only if visualized.

Some improvement has been included with the modularization reducing the number plugin in the homepage.

  • We could additionally think to use the lazy+suspence react approach to trigger requests for specific resources only when the l specific list is mounted

CDN: at the release we can evaluate to speed-up script loading by deploying and getting the new release bundles from a CDN.

We did not worked on this yet. Is this still relevant?

@tdipisa tdipisa added estimate needed and removed QF Quality Friday Priority: High labels Jun 14, 2024
@tdipisa
Copy link
Member Author

tdipisa commented Jun 14, 2024

The following are next steps to be evaluated and possibly implemented for 2024.02.00 (@allyoucanmap)

  • Deprecation/dead code removal
  • ogc-schema removal and refactor of the catalog tool for CSW
  • Investigation on lazy loading capabilities using React and review of the current module loading

@tdipisa tdipisa added this to the 2024.02.00 milestone Jun 14, 2024
allyoucanmap added a commit to allyoucanmap/MapStore2 that referenced this issue Jun 24, 2024
@allyoucanmap allyoucanmap linked a pull request Jun 24, 2024 that will close this issue
5 tasks
@allyoucanmap allyoucanmap linked a pull request Jun 24, 2024 that will close this issue
5 tasks
allyoucanmap added a commit to allyoucanmap/MapStore2 that referenced this issue Jul 4, 2024
@tdipisa tdipisa linked a pull request Jul 5, 2024 that will close this issue
5 tasks
@tdipisa tdipisa reopened this Jul 11, 2024
@tdipisa tdipisa linked a pull request Jul 22, 2024 that will close this issue
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment