Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable SDMX-ML SAX (Simple API for XML) parsing method #36

Open
eblondel opened this issue Oct 9, 2014 · 0 comments
Open

Enable SDMX-ML SAX (Simple API for XML) parsing method #36

eblondel opened this issue Oct 9, 2014 · 0 comments

Comments

@eblondel
Copy link
Member

eblondel commented Oct 9, 2014

When SDMX-ML documents become large, ending up with 2 copies of the data in memory at once, i.e the tree and the R datastructure (resulting from as.data.frame in rsdmx), could lead to memory issues.
In this case, event-driven or SAX (Simple API for XML) style parser could be investigated, not as replacement but as complementary approach to read SDMX-ML files. With the XML package, event-driven parsing is possible using xmlEventParser.

Under this activity, an enhancement of readSDMX should allow reading directly from URL (for SDMX web-services that do not require an explicit user-agent). Combined with SAX approach, it intends to provide a powerful SDMX reader that parse SDMX data from the web and minimizes the R in-memory not storing the SDMX-ML XML content, ideal for processing R web-services using rsdmx.

This activity is listed as priority enhancement for which rsdmx seeks for fundings. See https://github.com/opensdmx/rsdmx/wiki#8-fundings
.


Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

@eblondel eblondel self-assigned this Oct 9, 2014
@eblondel eblondel added the RnD label Oct 9, 2014
@eblondel eblondel changed the title Investigate alternative use of xmlEventParse Enable SDMX-ML SAX (Simple API for XML) parsing method Nov 21, 2014
@eblondel eblondel assigned eblondel and unassigned eblondel May 16, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant