fix: large memory consumption while parsing corrupted PE. #2120
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
After #2119 `yara` is consuming a large amount of memory while parsing 9bddb45c44d9c25a4f97ef800cb110de5e6a15349bac05d389c8bda37902f25a. That's because after the change it doesn't limit the total number of imported functions, only the total number of parsing attempts, but the count is reset with each import entry. This file has a large number of entries and a large number of functions per entry, the total number of functions is very high. It turns out that we must limit both the total number of correctly parsed functions (for cases like this one), and the total number of parsing attempts (for cases like the one #2119 was aiming to solve).
wxsBSD
approved these changes
Nov 28, 2024
DavidTurland
added a commit
to DavidTurland/yara
that referenced
this pull request
Dec 23, 2024
* Fix bug when parsing imports from 32bit PE. (VirusTotal#2119) There is a bug when parsing [1] which turns out to be incorrectly checking the number of successfully parsed imported functions and not the number of parsed attempts. This particular sample is badly malformed and is causing excessive loops in the parser while attempting to parse invalid data. With this fix it will finish in a few seconds on my laptop. [1]: 9c8e4dfa84b1ce7e919964978d33eada266d58b2aacdbef44b0618cc178ea421 * Upgrade `upload-artifact` action to v4 * fix: large memory consumption while parsing corrupted PE. (VirusTotal#2120) After VirusTotal#2119 `yara` is consuming a large amount of memory while parsing 9bddb45c44d9c25a4f97ef800cb110de5e6a15349bac05d389c8bda37902f25a. That's because after the change it doesn't limit the total number of imported functions, only the total number of parsing attempts, but the count is reset with each import entry. This file has a large number of entries and a large number of functions per entry, the total number of functions is very high. It turns out that we must limit both the total number of correctly parsed functions (for cases like this one), and the total number of parsing attempts (for cases like the one VirusTotal#2119 was aiming to solve). * Remove dotnet configure instructions since it's enabled by default now (VirusTotal#2122) * Upgrade to `actions/cache@v4`. GitHub is deprecating versions 2 and 3. * Fix typo in documentation. * Rename cache trying to solve issue with Bazel build * Disable cache in Bazel build workflow * Add Corelight to list of YARA users (VirusTotal#2124) --------- Co-authored-by: Wesley Shields <[email protected]> Co-authored-by: Victor M. Alvarez <[email protected]> Co-authored-by: zdwg42 <[email protected]> Co-authored-by: signalblur <[email protected]>
DavidTurland
added a commit
to DavidTurland/yara
that referenced
this pull request
Dec 23, 2024
* Fix bug when parsing imports from 32bit PE. (VirusTotal#2119) There is a bug when parsing [1] which turns out to be incorrectly checking the number of successfully parsed imported functions and not the number of parsed attempts. This particular sample is badly malformed and is causing excessive loops in the parser while attempting to parse invalid data. With this fix it will finish in a few seconds on my laptop. [1]: 9c8e4dfa84b1ce7e919964978d33eada266d58b2aacdbef44b0618cc178ea421 * Upgrade `upload-artifact` action to v4 * fix: large memory consumption while parsing corrupted PE. (VirusTotal#2120) After VirusTotal#2119 `yara` is consuming a large amount of memory while parsing 9bddb45c44d9c25a4f97ef800cb110de5e6a15349bac05d389c8bda37902f25a. That's because after the change it doesn't limit the total number of imported functions, only the total number of parsing attempts, but the count is reset with each import entry. This file has a large number of entries and a large number of functions per entry, the total number of functions is very high. It turns out that we must limit both the total number of correctly parsed functions (for cases like this one), and the total number of parsing attempts (for cases like the one VirusTotal#2119 was aiming to solve). * Remove dotnet configure instructions since it's enabled by default now (VirusTotal#2122) * Upgrade to `actions/cache@v4`. GitHub is deprecating versions 2 and 3. * Fix typo in documentation. * Rename cache trying to solve issue with Bazel build * Disable cache in Bazel build workflow * Add Corelight to list of YARA users (VirusTotal#2124) --------- Co-authored-by: Wesley Shields <[email protected]> Co-authored-by: Victor M. Alvarez <[email protected]> Co-authored-by: zdwg42 <[email protected]> Co-authored-by: signalblur <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
After #2119
yarais consuming a large amount of memory while parsing 9bddb45c44d9c25a4f97ef800cb110de5e6a15349bac05d389c8bda37902f25a. That's because after the change it doesn't limit the total number of imported functions, only the total number of parsing attempts, but the count is reset with each import entry. This file has a large number of entries and a large number of functions per entry, the total number of functions is very high.It turns out that we must limit both the total number of correctly parsed functions (for cases like this one), and the total number of parsing attempts (for cases like the one #2119 was aiming to solve).