Unable to change message format in SEQ

[matt-amwins]

Despite multiple attempts, I am unable to get the message formatting to match the style being used by the console. When I inspect the log formatter while debugging, I can see that the format looks correct. However, the message still comes up in SEQ with the default formatting. Appreciate any help.

Init File

import logging.handlers
import logging.config
import logging
from pathlib import Path
from copy import deepcopy
import socket
import seqlog
import yaml

def configure_logging(app, name=None):
    config = app.config
    try:
        logging.raiseExceptions = config["ENV"] in ["development", "local"]
        log_config = get_log_config_dict()
        log_config["handlers"]["seq"]["server_url"] = config.get("SEQ_SERVER_URL")
        log_config["handlers"]["seq"]["api_key"] = config.get("SEQ_API_KEY")
        seqlog.configure_from_dict(log_config, override_root_logger=True, use_structured_logger=True, support_extra_properties=True, support_stack_info=True, use_clef=False)
        seqlog.set_global_log_properties(
            Application="AMI",
            Environment=config.get("ENV", "development"),
            Hostname=socket.gethostname()
        )
        seqlog.set_callback_on_failure(
            lambda e: logging.exception('Failure occurred during log submission', exc_info=e),
        )
        logging.info("Seq logging configured")
        return logging.getLogger(name) or logging.getLogger()
    except Exception as e:
        print(f"Error configuring Seq logging: {e}")
        return logging.getLogger()

def get_log_config_dict():
    with open('log_config.yml', 'r') as log_config_file:
        return yaml.safe_load(log_config_file.read())

Logging Config

version: 1

disable_existing_loggers: True

root:
  level: INFO
  handlers:
  - console
  - seq

loggers:
  CELERY:
    level: INFO
    handlers:
    - console
    - seq
    propagate: no

handlers:
  console:
    class: seqlog.structured_logging.ConsoleStructuredLogHandler
    formatter: standard

  seq:
    class: seqlog.structured_logging.SeqLogHandler
    formatter: standard
    server_url: ''
    api_key: ''
    batch_size: 10
    auto_flush_timeout: 10

formatters:
  standard:
    format: '[%(asctime)s] - [%(process)d] [%(levelname)s]: %(filename)s %(funcName)s (%(lineno)d): %(message)s'
    datefmt: '%Y-%m-%d %H:%M:%S'
    style: '%'

Result

[tintoy]

Hi - I’ll take a proper look at this in the afternoon, but I suspect it’s because we don’t support mixing % and {NamedHole} formatting.

[tintoy]

Ah, I think I see what you are asking for. I suspect that may be difficult to implement because the Seq API wants us to supply a message template and arguments for each log entry, not a pre-formatted log message. We might be able to run the message template through the configured formatter before passing it to Seq but there may wind up being odd edge cases around escaping etc.

@piotrmaslanka - a second opinion would be appreciated; does what I’m saying make sense to you?

It looks like this is an issue when using either Ingest or CLEF.

[matt-amwins]

@tintoy appreciate you looking into this! One thing worth mentioning, I have CLEF disabled due to our SEQ instance being an older version (Seq 2022.1.7929). I get all kinds of errors when I turn it on.