-
Notifications
You must be signed in to change notification settings - Fork 512
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Extracts web domains and IP address and implements tests #2031
Open
aaronatp
wants to merge
23
commits into
mandiant:master
Choose a base branch
from
aaronatp:master
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit
Hold shift + click to select a range
1b6bdaa
This pull request extracts web domains and IP addresses from files an…
aaronatp 999f656
Update domain_ip_helpers.py
aaronatp 80fc4d4
Update extract_domain_and_ip.py
aaronatp 2bb09f8
Update test_domain_ip_extractor.py
aaronatp 61c4ad5
Adds to changelog and fixes string concatenation style error
aaronatp 627f187
Merge branch 'master' into master
aaronatp 9b41074
Help debug why it won't build with PyInstaller 3.11
aaronatp 6236b8f
Update capa/capabilities/extract_domain_and_ip.py
aaronatp f9f6bb4
Update capa/capabilities/extract_domain_and_ip.py
aaronatp aa5f542
Update verbose.py
aaronatp 2e9c1a9
Update vverbose.py
aaronatp 11585c3
Update extract_domain_and_ip.py
aaronatp 58b0336
Fix PyInstaller 3.11 installation error with get_signatures
aaronatp c55169b
Fix get_extractor_from_doc issue for CAPE
aaronatp 85dfa55
Fix 'backend' assignment in get_extractor_from_doc
aaronatp 5afd175
Fix 'format' in get_extractor_from_doc
aaronatp 78d0efc
Merge branch 'master' into master
aaronatp 681e6c3
Reformat imports
aaronatp fb3ed8a
Reformat multi-line string
aaronatp 0443caf
Correct Flake8 errors
aaronatp a000461
Update domain_ip_helpers.py
aaronatp 93943e5
Update domain_ip_helpers.py
aaronatp 32b4778
Update capa/render/verbose.py
aaronatp File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,84 @@ | ||
import logging | ||
from pathlib import Path | ||
|
||
from capa.helpers import get_auto_format | ||
from capa.features.common import FORMAT_CAPE | ||
from capa.render.result_document import ResultDocument | ||
from capa.features.extractors.base_extractor import FeatureExtractor | ||
from capa.features.extractors.cape.extractor import CapeExtractor | ||
|
||
logger = logging.getLogger(__name__) | ||
|
||
BACKEND_VIV = "vivisect" | ||
BACKEND_DOTNET = "dotnet" | ||
BACKEND_BINJA = "binja" | ||
BACKEND_PEFILE = "pefile" | ||
|
||
|
||
def get_file_path(doc: ResultDocument) -> Path: | ||
return Path(doc.meta.sample.path) | ||
|
||
|
||
def get_sigpaths_from_doc(doc: ResultDocument): | ||
import capa.loader | ||
|
||
if doc.meta.argv: | ||
try: | ||
if "-s" in list(doc.meta.argv): | ||
idx = doc.meta.argv.index("-s") | ||
sigpath = Path(doc.meta.argv[idx + 1]) | ||
if "./" in str(sigpath): | ||
fixed_str = str(sigpath).split("./")[1] | ||
sigpath = Path(fixed_str) | ||
|
||
elif "--signatures" in list(doc.meta.argv): | ||
idx = doc.meta.argv.index("--signatures") | ||
sigpath = Path(doc.meta.argv[idx + 1]) | ||
if "./" in str(sigpath): | ||
fixed_str = str(sigpath).split("./")[1] | ||
sigpath = Path(fixed_str) | ||
|
||
else: | ||
sigpath = "(embedded)" # type: ignore | ||
|
||
return capa.loader.get_signatures(sigpath) | ||
|
||
except AttributeError: | ||
raise NotImplementedError("Confirm that argv is an attribute of doc.meta") | ||
|
||
else: | ||
print("in 'get_sigpaths_from_doc', run in debug (-d) mode") | ||
logger.debug("'doc.meta' has not attribute 'argv', this is probably a bad sign...") | ||
|
||
|
||
def get_extractor_from_doc(doc: ResultDocument) -> FeatureExtractor: | ||
import capa.loader | ||
|
||
path = get_file_path(doc) | ||
format = doc.meta.analysis.format | ||
os = doc.meta.analysis.os | ||
|
||
_ = get_auto_format(get_file_path(doc)) | ||
if format == FORMAT_CAPE: | ||
report = capa.helpers.load_json_from_path(path) | ||
return CapeExtractor.from_report(report) | ||
elif _ == BACKEND_VIV: | ||
backend = BACKEND_VIV | ||
elif _ == BACKEND_PEFILE: | ||
backend = BACKEND_PEFILE | ||
elif _ == BACKEND_BINJA: | ||
backend = BACKEND_BINJA | ||
elif _ == BACKEND_DOTNET: | ||
backend = BACKEND_DOTNET | ||
else: | ||
backend = BACKEND_VIV # according to main.py this is the default | ||
|
||
sigpath = get_sigpaths_from_doc(doc) | ||
|
||
return capa.loader.get_extractor( | ||
input_path=path, | ||
input_format=format, | ||
os_=os, | ||
backend=backend, | ||
sigpaths=sigpath, | ||
) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this was for debugging? Why are you not using logger here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @VascoSch92 yes trying to figure out why it's failing to build with PyInstaller 3.11
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah ok easy... I just saw it and I wanted to check :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Haha yes thank you for the question though, I appreciate the comments and feedback you're giving me