You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When SDMX-ML documents become large, ending up with 2 copies of the data in memory at once, i.e the tree and the R datastructure (resulting from as.data.frame in rsdmx), could lead to memory issues.
In this case, event-driven or SAX (Simple API for XML) style parser could be investigated, not as replacement but as complementary approach to read SDMX-ML files. With the XML package, event-driven parsing is possible using xmlEventParser.
Under this activity, an enhancement of readSDMX should allow reading directly from URL (for SDMX web-services that do not require an explicit user-agent). Combined with SAX approach, it intends to provide a powerful SDMX reader that parse SDMX data from the web and minimizes the R in-memory not storing the SDMX-ML XML content, ideal for processing R web-services using rsdmx.
When SDMX-ML documents become large, ending up with 2 copies of the data in memory at once, i.e the tree and the R datastructure (resulting from
as.data.frame
inrsdmx
), could lead to memory issues.In this case, event-driven or SAX (Simple API for XML) style parser could be investigated, not as replacement but as complementary approach to read SDMX-ML files. With the
XML
package, event-driven parsing is possible usingxmlEventParser
.Under this activity, an enhancement of
readSDMX
should allow reading directly from URL (for SDMX web-services that do not require an explicit user-agent). Combined with SAX approach, it intends to provide a powerful SDMX reader that parse SDMX data from the web and minimizes the R in-memory not storing the SDMX-ML XML content, ideal for processing R web-services usingrsdmx
.This activity is listed as priority enhancement for which rsdmx seeks for fundings. See https://github.com/opensdmx/rsdmx/wiki#8-fundings
.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
The text was updated successfully, but these errors were encountered: