Merge pull request #303 from carbonblack/release-1.4.0

CBAPI-3883: Release 1.4.0 -> master
carbonblack · Jul 26, 2022 · 50ed599 · 50ed599
2 parents 885dd40 + 8dbc048
commit 50ed599
Show file tree

Hide file tree

Showing 74 changed files with 5,753 additions and 2,147 deletions.
diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -10,7 +10,7 @@ assignees:
 **I am seeing this behaviour on: (please complete the following information):**
  - OS: [e.g. iOS]
  - Carbon Black Cloud Products: [e.g. Endpoint Standard, Enterprise EDR, Audit & Remediation]
- - Python Version: [e.g. 3.6]
+ - Python Version: [e.g. 3.7]
 
 **Describe the bug**
 A clear and concise description of what the bug is.

diff --git a/README.md b/README.md
@@ -1,8 +1,8 @@
 # VMware Carbon Black Cloud Python SDK
 
-**Latest Version:** 1.3.6
+**Latest Version:** 1.4.0
 <br>
-**Release Date:** April 19, 2022
+**Release Date:** July 26, 2022
 
 [![Coverage Status](https://coveralls.io/repos/github/carbonblack/carbon-black-cloud-sdk-python/badge.svg?t=Id6Baf)](https://coveralls.io/github/carbonblack/carbon-black-cloud-sdk-python)
 [![Codeship Status for carbonblack/carbon-black-cloud-sdk-python](https://app.codeship.com/projects/9e55a370-a772-0138-aae4-129773225755/status?branch=develop)](https://app.codeship.com/projects/402767)
@@ -30,7 +30,7 @@ Visit [ReadTheDocs](https://carbon-black-cloud-python-sdk.readthedocs.io/en/late
 
 ## Requirements
 
-The Carbon Black Cloud Python SDK is design to work on Python 3.6 and above.
+The Carbon Black Cloud Python SDK is design to work on Python 3.7 and above.
 
 All requirements are installed as part of `pip install carbon-black-cloud-sdk`. If you're planning on pushing changes to the Carbon Black Cloud Python SDK, the following can be used after cloning the repo `pip install -r requirements.txt`
 
@@ -47,13 +47,8 @@ At least one Carbon Black Cloud product is required to use this SDK:
 ### Python Packages
 
 - requests
-- cachetools
 - pyyaml
-- pika
-- prompt_toolkit
-- pygments
 - python-dateutil
-- protobuf
 - schema
 - solrq
 - validators

diff --git a/VERSION b/VERSION
@@ -1 +1 @@
-1.3.6
+1.4.0
diff --git a/codeship-services.yml b/codeship-services.yml
@@ -1,7 +1,3 @@
-testingpython36:
-  build:
-    dockerfile: ./docker/python3.6/Dockerfile
-
 testingpython37:
   build:
     dockerfile: ./docker/python3.7/Dockerfile
@@ -12,6 +8,14 @@ testingpython38:
   encrypted_env_file:
     - env.encrypted
 
+testingpython39:
+  build:
+    dockerfile: ./docker/python3.9/Dockerfile
+
+testingpython310:
+  build:
+    dockerfile: ./docker/python3.10/Dockerfile
+
 testingrhel:
   build:
     dockerfile: ./docker/rhel/Dockerfile

diff --git a/codeship-steps.yml b/codeship-steps.yml
@@ -5,15 +5,18 @@
 - name: Tests
   type: parallel
   steps:
-  - name: testing python 3.6
-    service: testingpython36
-    command: pytest
   - name: testing python 3.7
     service: testingpython37
     command: pytest
   - name: testing python 3.8
     service: testingpython38
     command: bin/tests_n_reports.sh
+  - name: testing python 3.9
+    service: testingpython39
+    command: pytest
+  - name: testing python 3.10
+    service: testingpython310
+    command: pytest
   - name: testing red hat
     service: testingrhel
     command: pytest

diff --git a/docker/python3.6/Dockerfile → docker/python3.10/Dockerfile b/docker/python3.6/Dockerfile → docker/python3.10/Dockerfile
@@ -1,4 +1,4 @@
-from python:3.6.10
+from python:3.10
 MAINTAINER [email protected]
 
 COPY . /app

diff --git a/docker/python3.7/Dockerfile b/docker/python3.7/Dockerfile
@@ -1,4 +1,4 @@
-from python:3.7.7
+from python:3.7
 MAINTAINER [email protected]
 
 COPY . /app

diff --git a/docker/python3.8/Dockerfile b/docker/python3.8/Dockerfile
@@ -1,4 +1,4 @@
-from python:3.8.3
+from python:3.8
 MAINTAINER [email protected]
 
 COPY . /app

diff --git a/docker/python3.9/Dockerfile b/docker/python3.9/Dockerfile
@@ -0,0 +1,7 @@
+from python:3.9
+MAINTAINER [email protected]
+
+COPY . /app
+WORKDIR /app
+
+RUN pip3 install -r requirements.txt
diff --git a/docker/rhel/Dockerfile b/docker/rhel/Dockerfile
@@ -4,7 +4,7 @@ MAINTAINER [email protected]
 COPY . /app
 WORKDIR /app
 
-RUN dnf install -y redhat-rpm-config gcc libffi-devel python3-devel openssl-devel
+RUN dnf install -y redhat-rpm-config gcc libffi-devel python38-devel openssl-devel
 RUN pip3 install --upgrade pip
 RUN pip3 install -r requirements.txt
 RUN pip3 install .
diff --git a/docker/suse/Dockerfile b/docker/suse/Dockerfile
@@ -1,4 +1,4 @@
-from opensuse/leap
+from opensuse/tumbleweed
 MAINTAINER [email protected]
 
 COPY . /app

diff --git a/docker/ubuntu/Dockerfile b/docker/ubuntu/Dockerfile
@@ -1,4 +1,4 @@
-from ubuntu:18.04
+from ubuntu:20.04
 MAINTAINER [email protected]
 
 COPY . /app

diff --git a/docs/cbc_sdk.platform.rst b/docs/cbc_sdk.platform.rst
@@ -52,6 +52,14 @@ cbc\_sdk.platform.jobs module
    :undoc-members:
    :show-inheritance:
 
+cbc\_sdk.platform.policies module
+----------------------------------
+
+.. automodule:: cbc_sdk.platform.policies
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 cbc\_sdk.platform.processes module
 ----------------------------------
 

diff --git a/docs/changelog.rst b/docs/changelog.rst
@@ -1,5 +1,51 @@
 Changelog
 ================================
+CBC SDK 1.4.0 - Released July 26,2022
+-------------------------------------
+
+**Breaking Changes:**
+
+* ``Policy`` object has been moved from ``cbc_sdk.endpoint_standard`` to ``cbc_sdk.platform``, as it now uses the new
+  Policy Services API rather than the old APIs through Integration Services.
+
+  - **N.B.:** This change means that you *must* use a custom API key with permissions under ``org.policies`` to manage
+    policies, rather than an older "API key."
+  - To enable time to update integration logic, the ``cbc_sdk.endpoint_standard Policy`` object may still be imported
+    from the old package, and supports operations that are backwards-compatible with the old one.
+  - When developing a new integration, or updating an existing one cbc_sdk.platform should be used. There is a utility
+    class ``PolicyBuilder``, and as features are added to the Carbon Black Cloud, they will be added to this module.
+
+* Official support for Python 3.6 has been dropped, since that version is now end-of-life.  Added explicit testing
+  support for Python versions 3.9 and 3.10.  **N.B.:** End users should update their Python version to 3.7.x or
+  greater.
+
+New Features:
+
+* Credentials handler now supports OAuth tokens.
+* Added support for querying a single ``Report`` from a ``Feed``.
+* Added support for alert notes (create, delete, get, refresh).
+
+Updates:
+
+* Removed the (unused) ``revoked`` property from ``Grant`` objects.
+* Increased the asynchronous query thread pool to 3 threads by default.
+* Required version of ``lxml`` is now 4.9.1.
+* Added a user acceptance test script for Alerts.
+
+Bug Fixes:
+
+* Added ``max_rows`` to USB device query, fixing pagination.
+* Fixed an off-by-one error in Alerts Search resulting un duplicate alerts showing up in results.
+* Fixed an error in alert faceting operations due to sending excess input to the server.
+
+Documentation:
+
+* Watchlists, Feeds, and Reports guide has been updated with additional clarification and examples.
+* Updated description for some ``Device`` fields that are never populated.
+* Additional sensor states added to ``Device`` documentation.
+* Fixed the description of ``BaseAlertSearchQuery.set_types`` so that it mentions all valid alert types.
+* Threat intelligence example has been deprecated.
+
 CBC SDK 1.3.6 - Released April 19, 2022
 ---------------------------------------
 

diff --git a/docs/conf.py b/docs/conf.py
@@ -23,7 +23,7 @@
 author = 'Developer Relations'
 
 # The full version, including alpha/beta/rc tags
-release = '1.3.6'
+release = '1.4.0'
 
 
 # -- General configuration ---------------------------------------------------

diff --git a/docs/installation.rst b/docs/installation.rst
@@ -6,7 +6,7 @@ If you already have Python installed, skip to `Use Pip`_.
 Install Python
 --------------
 
-Carbon Black Cloud Python SDK is compatible with Python 3.6+.
+Carbon Black Cloud Python SDK is compatible with Python 3.7+.
 UNIX systems usually have Python installed by default; it will
 have to be installed on Windows systems separately.
 
@@ -19,7 +19,7 @@ at a command prompt::
     $ pip --version
     pip 20.2.3 from /usr/local/lib/python3.7/site-packages (python 3.7)
 
-If “python --version” reports back a version of 3.6.x or higher, you’re all set.
+If “python --version” reports back a version of 3.7.x or higher, you’re all set.
 If “pip” is not found, follow the instructions on this
 `guide <https://pip.pypa.io/en/stable/installing/>`_.
 

diff --git a/docs/requirements.txt b/docs/requirements.txt
@@ -1,3 +1,4 @@
 # Defining the exact version will make sure things don't break
 sphinxcontrib-apidoc
 sphinx-copybutton==0.4.0
+pygments
diff --git a/docs/watchlists-feeds-reports.rst b/docs/watchlists-feeds-reports.rst
@@ -191,6 +191,94 @@ Both IOCs and reports may include a ``link`` property, which is used by the Carb
 to indicate that this IOC or report is being managed outside of the console.  If this property is not ``None``,
 the console UI will disable the ability to edit the IOC or report, but they can still be edited via the API.
 
+Creating an IOC
++++++++++++++++
+
+You can create an IOC via the `IOC_V2` class, there are 3 avaliable methods that you can use to initiate your IOC:
+`IOC_V2.create_query, IOC_V2.create_equality, IOC_V2.create_regex`.
+
+Creating an equality IOC
+++++++++++++++++++++++++
+
+.. code-block:: python
+
+    >> from cbc_sdk import CBCloudAPI
+    >> from cbc_sdk.enterprise_edr import IOC_V2
+    >> cbcsdk = CBCloudAPI(profile="default")
+    >> IOC_V2.create_equality(cbcsdk, None, "netconn_domain", ["localhost.local"])
+    <cbc_sdk.enterprise_edr.threat_intelligence.IOC_V2: id ad361179-d586-4c99-af3e-821224cc0fd9> @ https://<CBCInstanceURL>
+
+
+Creating a query IOC
+++++++++++++++++++++
+
+.. code-block:: python
+
+    >> IOC_V2.create_query(cbcsdk, None, "{process_hash:098f6bcd4621d373cade4e832627b4f6}")
+    <cbc_sdk.enterprise_edr.threat_intelligence.IOC_V2: id 36d68cab-4739-4aa6-afcc-2921d2e5573e> @ https://<CBCInstanceURL>
+
+
+Creating a regex IOC
+++++++++++++++++++++
+
+.. code-block:: python
+
+    >> IOC_V2.create_regex(cbcsdk, None, "process_name", r"(^/usr/.*$)|(^/bin/.*$)")
+    <cbc_sdk.enterprise_edr.threat_intelligence.IOC_V2: id 5170a04c-bbfc-4449-b939-d5fc9f55d555> @ https://<CBCInstanceURL>
+
+
+Removing and adding an IOC from a Report
+++++++++++++++++++++++++++++++++++++++++
+
+If you want to remove an IOC from a report, you will need the IOC id and the report id.
+
+.. code-block:: python
+
+    >> from cbc_sdk.enterprise_edr import Report
+    >> ioc_id = "<ioc_id>"
+    >> report = cbcsdk.select(Report).where(id="<report_id>", feed_id="<feed_id>")[0]
+    <cbc_sdk.enterprise_edr.threat_intelligence.Report: id 1e69c54e-7cc9-41b8-9d1d-3fd59a003d8a> @ https://<CBCInstanceURL>
+    >> report.remove_iocs_by_id([ioc_id])
+    >> report.update()
+    <cbc_sdk.enterprise_edr.threat_intelligence.Report: id 1e69c54e-7cc9-41b8-9d1d-3fd59a003d8b> @ https://<CBCInstanceURL> (*)
+
+
+Adding the IOC into the report works the same way:
+
+.. code-block:: python
+
+    >> from cbc_sdk.enterprise_edr import Report, IOC_V2
+    >> ioc_id = "<ioc_id>"
+    >> report = cbcsdk.select(Report).where(id="<report_id>", feed_id="<feed_id>")[0]
+    <cbc_sdk.enterprise_edr.threat_intelligence.Report: id 1e69c54e-7cc9-41b8-9d1d-3fd59a003d8a> @ https://<CBCInstanceURL>
+    >> ioc = IOC_V2.create_regex(cbcsdk, None, "process_name", r"(^/usr/.*$)|(^/bin/.*$)")
+    >> report.append_iocs([ioc])
+    >> report.update()
+    <cbc_sdk.enterprise_edr.threat_intelligence.Report: id 1e69c54e-7cc9-41b8-9d1d-3fd59a003d8b> @ https://<CBCInstanceURL> (*)
+
+
+.. note::
+    Calling the `Report.save()` method after the insertion or removal of IOC does not update the report
+    and it's likely to result in a bad call to the API.
+
+
+If the report is in a watchlist instead of a feed then you have to get the appropriate watchlist and iterate over the reports.
+
+.. code-block:: python
+
+    >> from cbc_sdk.enterprise_edr import Watchlist, Report, IOC_V2
+    >> ioc_id = "<ioc_id>"
+    >> report_id = "<report_id>"
+    >> watchlist = cbcsdk.select(Watchlist, "<watchlist_id>")
+    <cbc_sdk.enterprise_edr.threat_intelligence.Watchlist: id <watchlist_id>> @ https://<CBCInstanceURL>
+    >> ioc = IOC_V2.create_regex(cbcsdk, None, "process_name", r"(^/usr/.*$)|(^/bin/.*$)")
+    >> reports = watchlist.reports
+    >> report = [report_ for report_ in reports if report_.id == report_id][0]
+    >> report.append_iocs([ioc])
+    >> report.update()
+    <cbc_sdk.enterprise_edr.threat_intelligence.Report: id 1e69c54e-7cc9-41b8-9d1d-3fd59a003d8b> @ https://<CBCInstanceURL> (*)
+
+
 Tips for Using IOCs
 +++++++++++++++++++
 * You can safely ignore certain fields in an IOC.  For example, fields like ``alert_id`` and ``process_guid`` will

diff --git a/examples/endpoint_standard/policy_operations.py b/examples/endpoint_standard/policy_operations.py
@@ -10,15 +10,26 @@
 # * WARRANTIES OR CONDITIONS OF MERCHANTABILITY, SATISFACTORY QUALITY,
 # * NON-INFRINGEMENT AND FITNESS FOR A PARTICULAR PURPOSE.
 
-"""Example script showing policy operations."""
+"""This sample has been deprecated.
+
+The endpoint standard policy module (cbc_sdk.endpoint_standard) was deprecated
+in July 2022 in line with the underlying APIs. A replacement example is
+included in ../platform/policy_service_crud_operations.py.
+
+The cbc_sdk.endpoint_standard module has been replaced by the
+cbc_sdk.platform.policies module and this example will be removed in a future
+version of the SDK.
+
+Example script showing policy operations.
+"""
 
 import sys
 import json
 import logging
 
 from cbc_sdk.errors import ServerError
 from cbc_sdk.helpers import build_cli_parser, get_cb_cloud_object
-from cbc_sdk.endpoint_standard import Policy
+from cbc_sdk.endpoint_standard import Policy  # This is the module that has been deprecated.
 
 log = logging.getLogger(__name__)
 
@@ -156,7 +167,7 @@ def replace_rule(cb, parser, args):
 def main():
     """Main function for the Policy Operations script."""
     parser = build_cli_parser("Policy operations")
-    commands = parser.add_subparsers(help="Policy commands", dest="command_name")
+    commands = parser.add_subparsers(help="This sample has been deprecated.  Policy commands", dest="command_name")
 
     commands.add_parser("list", help="List all configured policies")
 

diff --git a/examples/enterprise_edr/threat_intelligence/README.md b/examples/enterprise_edr/threat_intelligence/README.md
@@ -1,6 +1,8 @@
 # ThreatIntel Module
 Python3 module that can be used in the development of Threat Intelligence Connectors for the Carbon Black Cloud.
 
+> **_NOTE:_** This connector is deprecated, it is recommended to use the [Carbon Black Cloud Threat Intelligence Connector](https://github.com/carbonblack/carbon-black-cloud-threat-intelligence-connector) instead!
+
 ## Requirements
 
 The file `requirements.txt` contains a list of dependencies for this project. After cloning this repository, run the following command from the `examples/enterprise_edr/threat_intelligence` directory:

diff --git a/examples/enterprise_edr/threat_intelligence/requirements.txt b/examples/enterprise_edr/threat_intelligence/requirements.txt
@@ -2,7 +2,7 @@ cybox==2.1.0.18
 dataclasses>=0.6
 cabby==0.1.20
 stix==1.2.0.7
-lxml==4.6.5
+lxml==4.9.1
 urllib3>=1.24.2
 python_dateutil==2.8.1
 PyYAML==5.4