Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unusually long parsing times for not that long documents #579

Open
isoos opened this issue Jan 30, 2024 · 1 comment
Open

Unusually long parsing times for not that long documents #579

isoos opened this issue Jan 30, 2024 · 1 comment
Labels
P2 A bug or feature request we're likely to work on type-bug Incorrect behavior (everything from a crash to more subtle misbehavior)

Comments

@isoos
Copy link

isoos commented Jan 30, 2024

Some long markdown files may take several seconds to parse and process, e.g.:
https://raw.githubusercontent.com/Workiva/over_react/master/CHANGELOG.md

Using:

List<m.Node> _parseMarkdownSource(String source) {
  final document = m.Document(
      extensionSet: m.ExtensionSet.gitHubWeb,
      blockSyntaxes: m.ExtensionSet.gitHubWeb.blockSyntaxes);
  final lines = source.replaceAll('\r\n', '\n').split('\n');
  return document.parseLines(lines);
}

It also looks like the parsing time is not linear with the length of the document:

  • single document parse time: 100%
  • two halves parsed separately and joined after: 86.4%
  • four quarters parsed separately and joined after: 52.26%

I expected a somewhat linear time to take a 2x document to parse, but apparently it is not that way. Let's explore if this can be improved.

@srawlins
Copy link
Member

srawlins commented Feb 5, 2024

Interesting analysis, thank you!

@srawlins srawlins added type-bug Incorrect behavior (everything from a crash to more subtle misbehavior) P2 A bug or feature request we're likely to work on labels Feb 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 A bug or feature request we're likely to work on type-bug Incorrect behavior (everything from a crash to more subtle misbehavior)
Projects
None yet
Development

No branches or pull requests

2 participants