Application Logging (Console / OpenTelemetry / Serilog / Seq)

Note

This page describes the application/host logging layer only - i.e. where the standard Microsoft.Extensions.Logging events produced by ASP.NET Core itself and by the WebGIS internals end up. It is independent of the GeoService performance/exception logging switch (Api:logging-type) described in Configuration of the API application, which writes dedicated webgis_performance/webgis_exceptions records rather than generic application log events - the two layers can end up pointed at the same database server, just at different tables.

The Api, Cms and Portal hosts all share the same setup, so everything below applies to all three (adjust the file names accordingly, e.g. Cms/appsettings.json).

Important

MicrosoftGeoServiceRequestLogger (the microsoft IGeoServiceRequestLogger, i.e. the per-request “audit” log, as opposed to the performance timings) logs at LogLevel.Trace - the most verbose level, disabled by default. Logging:LogLevel:Default is normally Information, so nothing is written for this category unless it is explicitly overridden (raising only Default is not enough, since Trace is more verbose than Information):

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Api.Core.AppCode.Services.Logging.MicrosoftGeoServiceRequestLogger": "Trace"
    }
  }
}

This can go into appsettings.json, _config/logging.json (see below), or as the environment variable Logging__LogLevel__Api.Core.AppCode.Services.Logging.MicrosoftGeoServiceRequestLogger=Trace. If a specific provider (e.g. Console) has its own narrower Logging:<Provider>:LogLevel section, that provider needs the same override too.

Under the hood, WebGIS always routes logging through Serilog, which normally only obeys its own Serilog:MinimumLevel section and ignores Logging:LogLevel entirely; WebGIS bridges Logging:LogLevel into Serilog’s MinimumLevel automatically, so this standard ASP.NET Core section keeps working as documented. An explicit Serilog:MinimumLevel/Override entry for the same category always takes precedence over the bridged Logging:LogLevel value.

Console (default)

Out of the box, logs go to the console (stdout), formatted as plain text - this is the ASP.NET Core default and requires no configuration. Log levels are controlled the usual way via the Logging:LogLevel section in appsettings.json or environment variables (Logging__LogLevel__Default=Information, …).

For structured/JSON console output (useful when a container log collector like Filebeat/Fluent Bit/Promtail parses stdout), switch the console formatter:

{
  "Logging": {
    "Console": {
      "FormatterName": "json"
    }
  }
}

Serilog sinks (SQL Server / PostgreSQL / Seq)

For deployments that need logs written directly into a relational database - without standing up an OpenTelemetry Collector - Api, Cms and Portal additionally run Serilog alongside the default logging pipeline, so it augments rather than replaces OTLP/console logging. Serilog is purely config-driven: with no Serilog section present, nothing changes from the default behavior (console output only).

To add a sink, add a Serilog section to appsettings.json (or appsettings.Production.json, or environment variables) - or, equivalently, to _config/logging.json (see above).

SQL Server

{
  "ConnectionStrings": {
    "LogsDb": "Server=sql-host;Database=WebGisLogs;User Id=webgis;Password=***;TrustServerCertificate=True"
  },
  "Serilog": {
    "WriteTo": [
      {
        "Name": "MSSqlServer",
        "Args": {
          "connectionString": "LogsDb",
          "sinkOptions": {
            "tableName": "Logs",
            "autoCreateSqlTable": true
          }
        }
      }
    ]
  }
}

PostgreSQL

The PostgreSQL sink needs its configuration companion package to be loaded explicitly via Using, and - unlike MSSqlServer, which ships sensible default columns - it has no built-in defaults: a Columns section describing which columns to write and how must always be supplied. Omitting it fails fast at startup with:

System.InvalidOperationException: 'Columns' section not found in provided configuration path:

The Columns section must be a root-level key of the configuration - a sibling of Serilog/ConnectionStrings, not nested inside Serilog (it is easy to assume otherwise, since everything else Serilog-related lives under the Serilog section). If it needs to be placed elsewhere (e.g. to avoid a name clash), point at it explicitly via Serilog:WriteTo:Args:configurationPath.

{
  "ConnectionStrings": {
    "LogsDb": "Host=pg-host;Port=5432;Database=webgis_logs;Username=webgis;Password=***"
  },
  "Serilog": {
    "Using": [ "Serilog.Sinks.PostgreSQL.Configuration" ],
    "WriteTo": [
      {
        "Name": "PostgreSQL",
        "Args": {
          "connectionString": "LogsDb",
          "tableName": "logs",
          "needAutoCreateTable": true
        }
      }
    ]
  },
  "Columns": {
    "message": "RenderedMessageColumnWriter",
    "message_template": "MessageTemplateColumnWriter",
    "level": {
      "Name": "LevelColumnWriter",
      "Args": { "renderAsText": true, "dbType": "Varchar" }
    },
    "raise_date": "TimestampColumnWriter",
    "exception": "ExceptionColumnWriter",
    "properties": "LogEventSerializedColumnWriter"
  }
}

Both sinks are batching sinks (periodic bulk insert), so they have negligible impact on request latency.

Seq

Unlike the DB sinks above, Seq is not a relational database - it is a small, self-hostable log server with its own storage engine and UI (see Setting up Seq below for how to run it). The sink just needs a URL and (optionally) an API key - no table/columns to define:

{
  "Serilog": {
    "WriteTo": [
      {
        "Name": "Seq",
        "Args": {
          "serverUrl": "http://seq-host:5341",
          "apiKey": "***"
        }
      }
    ]
  }
}

This, too, is a batching sink with negligible latency impact, and can run alongside the SQL Server/PostgreSQL sink (e.g. DB for long-term retention, Seq for day-to-day search) - Serilog happily writes to any number of configured sinks at once.

Oracle

There is currently no well-maintained Serilog sink for Oracle, so a direct DB sink is not offered for Oracle. If the log backend is Oracle-only, use the OTLP path instead (e.g. an OpenTelemetry Collector with an Oracle/JDBC exporter, or export to an OTLP-native backend and query it there).

Viewing & analyzing logs

Which tool makes sense depends on which sink(s) are enabled:

  • Only the SQL Server/PostgreSQL sink, ad-hoc troubleshooting: a regular DB client is enough - no extra tool needed. Azure Data Studio/SSMS (SQL Server) or pgAdmin/DBeaver (PostgreSQL). The structured Properties/properties column is JSON (NVARCHAR/jsonb), queryable directly, e.g.:

    -- SQL Server
SELECT TOP 100 * FROM Logs
WHERE Level = 'Warning' AND JSON_VALUE(Properties, '$.RequestPath') LIKE '%GetMap%'
ORDER BY TimeStamp DESC;

-- PostgreSQL
SELECT * FROM logs
WHERE level = 'Warning' AND properties->>'RequestPath' LIKE '%GetMap%'
ORDER BY raise_date DESC LIMIT 100;

    This does not give full-text search, dashboards, alerting, or trace correlation though - it is fine for “quick lookup”, not for ongoing analysis.

  • Dashboards/alerting/search - recommended: Grafana (OSS, self-hostable, free). It can be pointed at what is already configured, with no need to add another sink:

    • Directly at the SQL Server/PostgreSQL sink table via Grafana’s built-in SQL data sources - dashboards/alerts on top of data already being written today.

    • At the OTLP path via Loki (logs) + Tempo (traces) + Prometheus/Mimir (metrics) - the de-facto open source stack for OpenTelemetry, and a natural fit since Api/Cms/ Portal already export OTLP once OTEL_EXPORTER_OTLP_ENDPOINT is set (see above). This also enables trace/log/metric correlation (e.g. drilling from a slow GetMap trace into its log lines), which the DB sinks alone cannot provide.

  • Simplest “just show me the logs” option, no Grafana setup: Seq - single Docker container, understands Serilog’s structured events natively, free for a single user/small team, and accepts OTLP directly too (see below for step-by-step setup).

  • Customer already runs a cloud/enterprise observability platform: point OTEL_EXPORTER_OTLP_ENDPOINT at it - Azure Monitor/Application Insights, AWS CloudWatch, Elastic/OpenSearch + Kibana, Datadog, New Relic, … all accept OTLP natively (directly or via an OpenTelemetry Collector), no code changes required.

Setting up Seq

Unlike Grafana, Seq has no “SQL data source” concept - it can only show events that were sent to it directly, either via OTLP or as a Serilog sink. It cannot simply be pointed at an already-populated Logs/logs table in SQL Server/PostgreSQL and browse that in place; if that is the goal (view/analyze what the DB sink already collects, without touching anything else), use Grafana’s built-in SQL data source against that table instead (see above) - no dual-write, no extra sink required.

If Seq’s own UI/search is still preferred, WebGIS needs to additionally send events to Seq - either as the Seq sink documented above (dual-write, alongside the existing SQL Server/PostgreSQL sink if any), or via OTLP as described below - both are already fully wired up.

No WebGIS-specific sink/package is needed for the OTLP route below - Seq natively implements OTLP ingestion, so that path is purely a matter of pointing the existing OTLP exporter at it.

  1. Run Seq (persists its data in <local path>, replace <password> with the initial admin password):

    docker run --name seq -d --restart unless-stopped \
  -e ACCEPT_EULA=Y \
  -e SEQ_FIRSTRUN_ADMINPASSWORD=<password> \
  -v <local path>:/data \
  -p 5341:80 \
  datalust/seq

  2. In the Seq UI (http://<seq-host>:5341), create a dedicated API key per app under Settings > API Keys (recommended, not required) - this makes it easy to tell Api/Cms/Portal traffic apart later in Data > Ingestion.

  3. Point WebGIS’s OTLP exporter at Seq’s OTLP endpoint - either as real environment variables, or via _config/logging.env (see above):

    OTEL_EXPORTER_OTLP_ENDPOINT=http://<seq-host>:5341/ingest/otlp
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_HEADERS=X-Seq-ApiKey=<api-key-from-step-2>

    Note the endpoint is Seq’s ingestion path (/ingest/otlp), not just the host root - the exporter appends /v1/logs//v1/traces itself. OTEL_EXPORTER_OTLP_HEADERS can be omitted if no API key was created in step 2.

    Alternatively - no traces, but structured events with full property fidelity and less config - add the Seq sink to _config/logging.json instead, using the same URL/API key.

  4. Restart Api/Cms/Portal (environment variables are only read at process startup, so _config/logging.env changes need a restart, unlike _config/logging.json which is picked up automatically). Logs and traces (including the GeoService request/performance spans described above) start showing up in the Seq UI immediately - with full trace/span correlation via the Trace menu on each event.

The SQL Server/PostgreSQL sink can keep running in parallel if it is already configured (e.g. for long-term retention/compliance) - Seq is simply an additional, independent consumer of the same telemetry.

Notes

  • Everything on this page (Serilog sinks, OTLP, Seq, Grafana, …) concerns where application log events go; it is independent of the Api:logging-type GeoService performance/exception logging switch described in Configuration of the API application, which - since it now also supports sqlserver/postgres/sqlite/oracle - can end up pointed at the same database server, just a different, purpose-built webgis_performance/webgis_exceptions table rather than the generic Logs table the Serilog DB sinks write to.

  • Multiple sinks can be active at once (e.g. console + OTLP + SQL Server) - enable only what is needed.