Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Python: Modelling of the Standard Library #16840

Merged
merged 15 commits into from
Aug 12, 2024
Merged

Python: Modelling of the Standard Library #16840

Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
2118f23
Python: model optparse.OptionParser.parse_arg
yoff Jun 25, 2024
501cda4
Python: model `fnmatch.filter`
yoff Jun 25, 2024
bc55117
Python: model `copy.deepcopy` as a value step
yoff Jun 25, 2024
bdc4808
Python: MaD summary models
yoff Jun 25, 2024
eb32cbe
Python: codecs.open
yoff Jun 25, 2024
571be8b
Python: model more loggers
yoff Jun 25, 2024
b261145
Python: fix compilation
yoff Jun 26, 2024
a3076f4
Python: fix test expectations, add missing sanitizer
yoff Jun 26, 2024
bbc3ff2
Apply suggestions from code review
yoff Jun 28, 2024
59f9532
Python: remove strange sink
yoff Jun 28, 2024
5ddfe75
Python: Add value steps for sequence elements
yoff Jun 28, 2024
77a0087
Python: add tests for loggers
yoff Jun 28, 2024
e40ae2e
Python: adjust test expectations
yoff Jun 28, 2024
e30f725
Python: Remove questionable model for `multiprocessing.connection.Lis…
yoff Jul 22, 2024
3434c38
Python: update test expectations
yoff Jul 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions python/ql/lib/semmle/python/frameworks/Stdlib.qll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,10 +254,14 @@ module Stdlib {
* See https://docs.python.org/3.9/library/logging.html#logging.Logger.
*/
module Logger {
private import semmle.python.dataflow.new.internal.DataFlowDispatch as DD

/** Gets a reference to the `logging.Logger` class or any subclass. */
API::Node subclassRef() {
result = API::moduleImport("logging").getMember("Logger").getASubclass*()
or
result = API::moduleImport("logging").getMember("getLoggerClass").getReturn().getASubclass*()
or
result = ModelOutput::getATypeNode("logging.Logger~Subclass").getASubclass*()
}

Expand All @@ -277,6 +281,13 @@ module Stdlib {
ClassInstantiation() {
this = subclassRef().getACall()
or
this =
DD::selfTracker(subclassRef()
.getAValueReachableFromSource()
.asExpr()
.(ClassExpr)
.getInnerScope())
or
Comment on lines +284 to +290
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you add a test indicating we can now do this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah good point...this is one bit I am somewhat unsure of, do we want this? Also, it should trigger a performance check.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you added this in 77a0087 -- I think a DCA performance check for this whole PR is in place yes 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have set off a standard nightly/nightly DCA run. I did not include any extra options, so it should still extract the standard library; thus we should see a slight slowdown from the new modelling and we can gauge that it is not huge. (If we did stop extraction, that speedup would drown out what we are looking for.)

this = API::moduleImport("logging").getMember("root").asSource()
or
this = API::moduleImport("logging").getMember("getLogger").getACall()
Expand Down
Loading