Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #76 #78

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
14 changes: 12 additions & 2 deletions HISTORY.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,18 @@
History
=======

Current (0.4.0) - (unreleased yet)
-----------------------------------

0.5.0 - (2024-12-20)
--------------------

* made the configuration a little bit more Pythonic
* FeatureFlags were removed
* `configure_from_dict` is the official way to configure Seq
* `support_stack_info` was removed from `log_to_console`
* you can choose whether ConsoleStructuredLogHandler will log to stdout or to stderr.

0.4.0 - (2024-12-8)
--------------------

* You can enable and disable all of the feature flags at runtime
* Added support for the `CLEF submission format <https://docs.datalust.co/docs/posting-raw-events>`_.
Expand Down
10 changes: 10 additions & 0 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,16 @@
# The master toctree document.
master_doc = 'index'

autodoc_default_options = {

}
autodoc_default_flags = [
'show-inheritance'
]
autodoc_typehints = "description"
autoclass_content = 'both'


# General information about the project.
project = u'SeqLog'
copyright = u"2016, Adam Friedman"
Expand Down
22 changes: 0 additions & 22 deletions docs/feature_flags.rst

This file was deleted.

2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ Contents:

readme
installation
migration
usage
usage-gunicorn
Modules <api/modules>
feature_flags
contributing
authors
history
Expand Down
17 changes: 17 additions & 0 deletions docs/migration.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
===============================
Migration guide from 0.4 to 0.5
===============================

First of all, the official way to configure Seq is via :func:`seqlog.configure_from_dict`

Alternatively you can call :func:`seqlog.configure_from_file`.

.. warning:: DO NOT call :code:`logging.config.fromDict` directly.

Then, FeatureFlags were completely obliterated and moved to SeqLogHandler's constructor.

Then, the SeqLogHandler accepts way more arguments, that you can define in the dictionary supporting the configuration:

.. autoclass:: seqlog.structured_logging.SeqLogHandler


4 changes: 3 additions & 1 deletion docs/usage-gunicorn.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
Usage (Gunicorn)
================


Using seqlog with `Gunicorn <https://gunicorn.org/>` involves some additional configuration because of the way Gunicorn uses ``fork`` to create new worker processes.

A custom ``JSONEncoder`` is also used to handle objects that are not `JSON serializable`.
Expand Down Expand Up @@ -118,6 +119,7 @@ A custom ``JSONEncoder`` is also used to handle objects that are not `JSON seria
console:
class: seqlog.structured_logging.ConsoleStructuredLogHandler
formatter: standard
use_stdout: False

seq:
class: seqlog.structured_logging.SeqLogHandler
Expand Down Expand Up @@ -148,4 +150,4 @@ A custom ``JSONEncoder`` is also used to handle objects that are not `JSON seria
try:
return json.JSONEncoder.default(self, obj)
except:
return str(obj)
return str(obj)
19 changes: 17 additions & 2 deletions docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,16 @@
Usage
=====

Recommended way is to use

.. autofunction:: seqlog.configure_from_dict

or

.. autofunction:: seqlog.configure_from_file

This way you can leverage the Python logging configuration syntax.

Configure logging programmatically
----------------------------------

Expand Down Expand Up @@ -56,6 +66,8 @@ First, create your configuration file (e.g. ``/foo/bar/my_config.yml``):

# Configure logging from scratch.
disable_existing_loggers: True
override_root_logger: True
use_structured_logger: True

# Configure the root logger to use Seq
root:
Expand All @@ -78,6 +90,7 @@ First, create your configuration file (e.g. ``/foo/bar/my_config.yml``):
console:
class: seqlog.structured_logging.ConsoleStructuredLogHandler
formatter: seq
override_existing_logger: True

# Log to Seq
seq:
Expand All @@ -95,7 +108,7 @@ First, create your configuration file (e.g. ``/foo/bar/my_config.yml``):
seq:
style: '{'

Then, call ``seqlog.configure_from_file()``:
Then, call :func:`seqlog.configure_from_file`:

.. code-block:: python

Expand All @@ -114,6 +127,7 @@ Configuring logging from a dictionary

Seqlog can also use a dictionary to describe the desired logging configuration.
This dictionary has the schema specified in Python's `logging.config <https://docs.python.org/3/library/logging.config.html#logging-config-dictschema>`_ module.
With some extra options described in :func:`seqlog.configure_from_dict`.

.. code-block:: python

Expand All @@ -131,6 +145,7 @@ This dictionary has the schema specified in Python's `logging.config <https://do
another_logger = logging.getLogger('another_logger')
another_logger.info('This is another logger.')


Batching and auto-flush
-----------------------

Expand Down Expand Up @@ -197,7 +212,7 @@ If the callable returns None, it won't be added.
Note that some properties get different treatment if the CLEF mode is enabled.

Note that there is a short list of these, these won't be attached to Properties. They will get removed from there and
attached according to the `CLEF<https://clef-json.org/>`_ format:
attached according to the `CLEF <https://clef-json.org/>`_ format:

* ``span_id`` - this will get removed and be replaced with ``@sp``
* ``trace_id`` - this will get removed and be replaced with ``@tr``
Expand Down
105 changes: 23 additions & 82 deletions seqlog/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,11 @@
import logging
import logging.config
import typing
import warnings

import yaml

from seqlog.feature_flags import FeatureFlag, configure_feature
from seqlog.structured_logging import StructuredLogger, StructuredRootLogger
from seqlog.structured_logging import StructuredLogger, StructuredRootLogger, _override_root_logger
from seqlog.structured_logging import SeqLogHandler, ConsoleStructuredLogHandler
from seqlog.structured_logging import get_global_log_properties as _get_global_log_properties
from seqlog.structured_logging import set_global_log_properties as _set_global_log_properties
Expand All @@ -16,79 +17,37 @@

__author__ = 'Adam Friedman'
__email__ = '[email protected]'
__version__ = '0.4.0a1'
__version__ = '0.5.0'


def configure_from_file(file_name, override_root_logger=True, support_extra_properties=False, support_stack_info=False, ignore_seq_submission_errors=False,
use_clef=False):
def configure_from_file(file_name):
"""
Configure Seq logging using YAML-format configuration file.

Uses `logging.config.dictConfig()`.

:param file_name: The name of the configuration file to use.
:type file_name: str
:param override_root_logger: Override the root logger to use a Seq-specific implementation? (default: True)
:type override_root_logger: bool
:param support_extra_properties: Support passing of additional properties to log via the `extra` argument?
:type support_extra_properties: bool
:param support_stack_info: Support attaching of stack-trace information (if available) to log records?
:type support_stack_info: bool
:param ignore_seq_submission_errors: Ignore errors encountered while sending log records to Seq?
:type ignore_seq_submission_errors: bool
:param use_clef: use the newer submission format CLEF
:type use_clef: bool
Configure Seq logging using YAML-format configuration file. Essentially loads the YAML, and invokes
:func:`configure_from_dict`.
"""

configure_feature(FeatureFlag.EXTRA_PROPERTIES, support_extra_properties)
configure_feature(FeatureFlag.STACK_INFO, support_stack_info)
configure_feature(FeatureFlag.IGNORE_SEQ_SUBMISSION_ERRORS, ignore_seq_submission_errors)
configure_feature(FeatureFlag.USE_CLEF, use_clef)

with open(file_name) as config_file:
config = yaml.load(config_file, Loader=yaml.SafeLoader)

configure_from_dict(config, override_root_logger, True)
configure_from_dict(config)


def configure_from_dict(config, override_root_logger=True, use_structured_logger=True, support_extra_properties=None,
support_stack_info=None, ignore_seq_submission_errors=None,
use_clef=None):
def configure_from_dict(config):
"""
Configure Seq logging using a dictionary.
Configure Seq logging using a dictionary. Use it instead of logging.config.dictConfig().

Uses `logging.config.dictConfig()`.
Extra parameters you can specify (as dictionary keys).

Note that if you provide None to any of the default arguments, it just won't get changed (ie. it will stay the same).
* `use_structured_logger` - this will configure Python logging environment to use a StructuredLogger, ie. one that
understands keyword arguments
* `override_root_logger` - overrides root logger with a StructuredLogger

:param config: A dict containing the configuration.
:type config: dict
:param override_root_logger: Override the root logger to use a Seq-specific implementation? (default: True)
:type override_root_logger: bool
:param use_structured_logger: Configure the default logger class to be StructuredLogger, which support named format arguments? (default: True)
:type use_structured_logger: bool
:param support_extra_properties: Support passing of additional properties to log via the `extra` argument?
:type support_extra_properties: bool
:param support_stack_info: Support attaching of stack-trace information (if available) to log records?
:type support_stack_info: bool
:param ignore_seq_submission_errors: Ignore errors encountered while sending log records to Seq?
:type ignore_seq_submission_errors: bool
:param use_clef: use the newer submission format CLEF
:type use_clef: bool
The rest will be passed onto logging.config.dictConfig()
"""

configure_feature(FeatureFlag.EXTRA_PROPERTIES, support_extra_properties)
configure_feature(FeatureFlag.STACK_INFO, support_stack_info)
configure_feature(FeatureFlag.IGNORE_SEQ_SUBMISSION_ERRORS, ignore_seq_submission_errors)
configure_feature(FeatureFlag.USE_CLEF, use_clef)

if override_root_logger:
_override_root_logger()

# Must use StructuredLogger to support named format argments.
if use_structured_logger:
if config.pop('use_structured_logger', False):
logging.setLoggerClass(StructuredLogger)

if config.pop('override_root_logger', False):
_override_root_logger()
logging.config.dictConfig(config)


Expand Down Expand Up @@ -126,19 +85,15 @@ def log_to_seq(server_url, api_key=None, level=logging.WARNING,
:return: The `SeqLogHandler` that sends events to Seq. Can be used to forcibly flush records to Seq.
:rtype: SeqLogHandler
"""

configure_feature(FeatureFlag.EXTRA_PROPERTIES, support_extra_properties)
configure_feature(FeatureFlag.STACK_INFO, support_stack_info)
configure_feature(FeatureFlag.IGNORE_SEQ_SUBMISSION_ERRORS, ignore_seq_submission_errors)
configure_feature(FeatureFlag.USE_CLEF, use_clef)

logging.setLoggerClass(StructuredLogger)

if override_root_logger:
_override_root_logger()

log_handlers = [
SeqLogHandler(server_url, api_key, batch_size, auto_flush_timeout, json_encoder_class)
SeqLogHandler(server_url, api_key, batch_size, auto_flush_timeout, json_encoder_class,
support_extra_properties=support_extra_properties, support_stack_info=support_stack_info,
use_clef=use_clef, ignore_seq_submission_errors=ignore_seq_submission_errors)
]

if additional_handlers:
Expand All @@ -155,7 +110,7 @@ def log_to_seq(server_url, api_key=None, level=logging.WARNING,
return log_handlers[0]


def log_to_console(level=logging.WARNING, override_root_logger=False, support_extra_properties=False, support_stack_info=False, **kwargs):
def log_to_console(level=logging.WARNING, override_root_logger=False, support_extra_properties=False, **kwargs):
"""
Configure the logging system to send log entries to the console.

Expand All @@ -167,13 +122,8 @@ def log_to_console(level=logging.WARNING, override_root_logger=False, support_ex
when using the logging.XXX functions.
:param support_extra_properties: Support passing of additional properties to log via the `extra` argument?
:type support_extra_properties: bool
:param support_stack_info: Support attaching of stack-trace information (if available) to log records?
:type support_stack_info: bool
"""

configure_feature(FeatureFlag.EXTRA_PROPERTIES, support_extra_properties)
configure_feature(FeatureFlag.STACK_INFO, support_stack_info)

logging.setLoggerClass(StructuredLogger)

if override_root_logger:
Expand All @@ -182,7 +132,7 @@ def log_to_console(level=logging.WARNING, override_root_logger=False, support_ex
logging.basicConfig(
style='{',
handlers=[
ConsoleStructuredLogHandler()
ConsoleStructuredLogHandler(support_extra_properties=support_extra_properties)
],
level=level,
**kwargs
Expand Down Expand Up @@ -237,12 +187,3 @@ def clear_global_log_properties():

_clear_global_log_properties()


def _override_root_logger():
"""
Override the root logger with a `StructuredRootLogger`.
"""

logging.root = StructuredRootLogger(logging.WARNING)
logging.Logger.root = logging.root
logging.Logger.manager = logging.Manager(logging.Logger.root)
Loading
Loading