> ## Documentation Index
> Fetch the complete documentation index at: https://private-7c7dfe99-mintlify-8a08bda2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Monitoring Nginx logs with ClickStack

> Monitoring Nginx with ClickStack

export const TrackedLink = ({href, eventName, children, ...rest}) => {
  const handleClick = () => {
    try {
      if (typeof window !== "undefined" && window.galaxy && eventName) {
        window.galaxy.track(eventName, {
          interaction: "click"
        });
      }
    } catch (e) {}
  };
  return <a href={href} onClick={handleClick} {...rest}>
      {children}
    </a>;
};

export const Image = ({img, alt, size}) => {
  return <Frame>
      <img src={img} alt={alt} />
    </Frame>;
};

<Info>
  **TL;DR**

  Collect and visualize Nginx access logs (JSON format) in ClickStack using the OTel `filelog` receiver. Includes a demo dataset and pre-built dashboard.
</Info>

<h2 id="existing-nginx">
  Integration with existing Nginx
</h2>

This section covers configuring your existing Nginx installation to send logs to ClickStack by modifying the ClickStack OTel collector configuration.
If you would like to test the integration before configuring your own existing setup, you can test with our preconfigured setup and sample data in the [following section](/clickstack/integration-examples/nginx-logs#demo-dataset).

<h5 id="prerequisites">
  Prerequisites
</h5>

* ClickStack instance running
* Existing Nginx installation
* Access to modify Nginx configuration files

<Steps>
  <Step>
    <h4 id="configure-nginx">
      Configure Nginx log format
    </h4>

    First, configure Nginx to output logs in JSON format for easier parsing. Add this log format definition to your nginx.conf:

    The `nginx.conf` file is typically located at:

    * **Linux (apt/yum)**: `/etc/nginx/nginx.conf`
    * **macOS (Homebrew)**: `/usr/local/etc/nginx/nginx.conf` or `/opt/homebrew/etc/nginx/nginx.conf`
    * **Docker**: Configuration is usually mounted as a volume

    Add this log format definition to the `http` block:

    ```nginx theme={null}
    http {
        log_format json_combined escape=json
        '{'
          '"time_local":"$time_local",'
          '"remote_addr":"$remote_addr",'
          '"request_method":"$request_method",'
          '"request_uri":"$request_uri",'
          '"status":$status,'
          '"body_bytes_sent":$body_bytes_sent,'
          '"request_time":$request_time,'
          '"upstream_response_time":"$upstream_response_time",'
          '"http_referer":"$http_referer",'
          '"http_user_agent":"$http_user_agent"'
        '}';

        access_log /var/log/nginx/access.log json_combined;
        error_log /var/log/nginx/error.log warn;
    }
    ```

    After making this change, reload Nginx.
  </Step>

  <Step>
    <h4 id="custom-otel">
      Create custom OTel collector configuration
    </h4>

    ClickStack allows you to extend the base OpenTelemetry Collector configuration by mounting a custom configuration file and setting an environment variable. The custom configuration is merged with the base configuration managed by HyperDX via OpAMP.

    Create a file named nginx-monitoring.yaml with the following configuration:

    ```yaml theme={null}
    receivers:
      filelog:
        include:
          - /var/log/nginx/access.log
          - /var/log/nginx/error.log
        start_at: end 
        operators:
          - type: json_parser
            parse_from: body
            parse_to: attributes
          - type: time_parser
            parse_from: attributes.time_local
            layout: '%d/%b/%Y:%H:%M:%S %z'
          - type: add
            field: attributes.source
            value: "nginx"

    service:
      pipelines:
        logs/nginx:
          receivers: [filelog]
          processors:
            - memory_limiter
            - transform
            - batch
          exporters:
            - clickhouse
    ```

    This configuration:

    * Reads Nginx Logs from their standard locations
    * Parses JSON log entries
    * Extracts and preserves the original log timestamps
    * Adds source: Nginx attribute for filtering in HyperDX
    * Routes logs to the ClickHouse exporter via a dedicated pipeline

    <Note>
      - You only define new receivers and pipelines in the custom config
      - The processors (memory\_limiter, transform, batch) and exporters (clickhouse) are already defined in the base ClickStack configuration - you just reference them by name
      - The time\_parser operator extracts timestamps from Nginx's time\_local field to preserve original log timing
      - The pipelines route data from your receivers to the ClickHouse exporter via the existing processors
    </Note>
  </Step>

  <Step>
    <h4 id="load-custom">
      Configure ClickStack to load custom configuration
    </h4>

    To enable custom collector configuration in your existing ClickStack deployment, you must:

    1. Mount the custom config file at /etc/otelcol-contrib/custom.config.yaml
    2. Set the environment variable CUSTOM\_OTELCOL\_CONFIG\_FILE=/etc/otelcol-contrib/custom.config.yaml
    3. Mount your Nginx log directories so the collector can read them

    <h5 id="docker-compose">
      Option 1: Docker Compose
    </h5>

    Update your ClickStack deployment configuration:

    ```yaml theme={null}
    services:
      clickstack:
        # ... existing configuration ...
        environment:
          - CUSTOM_OTELCOL_CONFIG_FILE=/etc/otelcol-contrib/custom.config.yaml
          # ... other environment variables ...
        volumes:
          - ./nginx-monitoring.yaml:/etc/otelcol-contrib/custom.config.yaml:ro
          - /var/log/nginx:/var/log/nginx:ro
          # ... other volumes ...
    ```

    <h5 id="all-in-one">
      Option 2: Docker Run (All-in-One Image)
    </h5>

    If using the all-in-one image with docker run:

    ```bash theme={null}
    docker run --name clickstack \
      -p 8080:8080 -p 4317:4317 -p 4318:4318 \
      -e CUSTOM_OTELCOL_CONFIG_FILE=/etc/otelcol-contrib/custom.config.yaml \
      -v "$(pwd)/nginx-monitoring.yaml:/etc/otelcol-contrib/custom.config.yaml:ro" \
      -v /var/log/nginx:/var/log/nginx:ro \
      clickhouse/clickstack-all-in-one:latest
    ```

    <Note>
      Ensure the ClickStack collector has appropriate permissions to read the nginx log files. In production, use read-only mounts (:ro) and follow the principle of least privilege.
    </Note>
  </Step>

  <Step>
    <h4 id="verifying-logs">
      Verifying Logs in HyperDX
    </h4>

    Once configured, log into HyperDX and verify logs are flowing:

    1. Navigate to the search view
    2. Set source to Logs, and verify you see log entries with fields like request, request\_time, upstream\_response\_time, etc.

    This is an example of what you should see:

    <Image img="https://mintcdn.com/private-7c7dfe99-mintlify-8a08bda2/OwB6o9ddvLojEP8N/images/clickstack/nginx-logs-search-view.png?fit=max&auto=format&n=OwB6o9ddvLojEP8N&q=85&s=536a442af721d7b2ff2aa072b83e2be9" alt="Log view" width="3838" height="1938" data-path="images/clickstack/nginx-logs-search-view.png" />

    <Image img="https://mintcdn.com/private-7c7dfe99-mintlify-8a08bda2/OwB6o9ddvLojEP8N/images/clickstack/log-view.png?fit=max&auto=format&n=OwB6o9ddvLojEP8N&q=85&s=c04d7714d99e60030332a3c473d0a0b0" alt="Log view" width="3838" height="1934" data-path="images/clickstack/log-view.png" />
  </Step>
</Steps>

<h2 id="demo-dataset">
  Demo dataset
</h2>

For users who want to test the nginx integration before configuring their production systems, we provide a sample dataset of pre-generated nginx access logs with realistic traffic patterns.

<Steps>
  <Step>
    <h4 id="download-sample">
      Download the sample dataset
    </h4>

    ```bash theme={null}
    # Download the logs
    curl -O https://datasets-documentation.s3.eu-west-3.amazonaws.com/clickstack-integrations/access.log
    ```

    The dataset includes:

    * Log entries with realistic traffic patterns
    * Various endpoints and HTTP methods
    * Mix of successful requests and errors
    * Realistic response times and byte counts
  </Step>

  <Step>
    <h4 id="test-config">
      Create test collector configuration
    </h4>

    Create a file named `nginx-demo.yaml` with the following configuration:

    ```yaml theme={null}
    cat > nginx-demo.yaml << 'EOF'
    receivers:
      filelog:
        include:
          - /tmp/nginx-demo/access.log
        start_at: beginning  # Read from beginning for demo data
        operators:
          - type: json_parser
            parse_from: body
            parse_to: attributes
          - type: time_parser
            parse_from: attributes.time_local
            layout: '%d/%b/%Y:%H:%M:%S %z'
          - type: add
            field: attributes.source
            value: "nginx-demo"

    service:
      pipelines:
        logs/nginx-demo:
          receivers: [filelog]
          processors:
            - memory_limiter
            - transform
            - batch
          exporters:
            - clickhouse
    EOF
    ```
  </Step>

  <Step>
    <h4 id="run-demo">
      Run ClickStack with demo configuration
    </h4>

    Run ClickStack with the demo logs and configuration:

    ```bash theme={null}
    docker run --name clickstack-demo \
      -p 8080:8080 -p 4317:4317 -p 4318:4318 \
      -e CUSTOM_OTELCOL_CONFIG_FILE=/etc/otelcol-contrib/custom.config.yaml \
      -v "$(pwd)/nginx-demo.yaml:/etc/otelcol-contrib/custom.config.yaml:ro" \
      -v "$(pwd)/access.log:/tmp/nginx-demo/access.log:ro" \
      clickhouse/clickstack-all-in-one:latest
    ```
  </Step>

  <Step>
    <h4 id="verify-demo-logs">
      Verify logs in HyperDX
    </h4>

    Once ClickStack is running:

    1. Open [HyperDX](http://localhost:8080/) and log in to your account (you may need to create an account first)
    2. Navigate to the Search view and set the source to `Logs`
    3. Set the time range to **2025-10-19 11:00:00 - 2025-10-22 11:00:00**

    Here's what you should see in your search view:

    <Info>
      **Timezone Display**

      HyperDX displays timestamps in your browser's local timezone. The demo data spans 2025-10-20 11:00:00 - 2025-10-21 11:00:00 UTC. The wide time range ensures you'll see the demo logs regardless of your location. Once you see the logs, you can narrow the range to a 24-hour period for clearer visualizations.
    </Info>

    <Image img="https://mintcdn.com/private-7c7dfe99-mintlify-8a08bda2/OwB6o9ddvLojEP8N/images/clickstack/nginx-logs-search-view.png?fit=max&auto=format&n=OwB6o9ddvLojEP8N&q=85&s=536a442af721d7b2ff2aa072b83e2be9" alt="Log view" width="3838" height="1938" data-path="images/clickstack/nginx-logs-search-view.png" />

    <Image img="https://mintcdn.com/private-7c7dfe99-mintlify-8a08bda2/OwB6o9ddvLojEP8N/images/clickstack/log-view.png?fit=max&auto=format&n=OwB6o9ddvLojEP8N&q=85&s=c04d7714d99e60030332a3c473d0a0b0" alt="Log view" width="3838" height="1934" data-path="images/clickstack/log-view.png" />
  </Step>
</Steps>

<h2 id="dashboards">
  Dashboards and visualization
</h2>

To help you get started monitoring nginx with ClickStack, we provide essential visualizations for Nginx Logs.

<Steps>
  <Step>
    <h4 id="download">
      <TrackedLink href={'/examples/nginx-logs-dashboard.json'} download="nginx-logs-dashboard.json" eventName="docs.nginx_logs_monitoring.dashboard_download">Download</TrackedLink> the dashboard configuration
    </h4>
  </Step>

  <Step>
    <h4 id="import-dashboard">
      Import the pre-built dashboard
    </h4>

    1. Open HyperDX and navigate to the Dashboards section.
    2. Click "Import Dashboard" in the upper right corner under the ellipses.

    <Image img="https://mintcdn.com/private-7c7dfe99-mintlify-8a08bda2/OwB6o9ddvLojEP8N/images/clickstack/import-dashboard.png?fit=max&auto=format&n=OwB6o9ddvLojEP8N&q=85&s=cdfe26f160c1c080b995c8451311241d" alt="Import Dashboard" width="3024" height="556" data-path="images/clickstack/import-dashboard.png" />

    3. Upload the nginx-logs-dashboard.json file and click finish import.

    <Image img="https://mintcdn.com/private-7c7dfe99-mintlify-8a08bda2/OwB6o9ddvLojEP8N/images/clickstack/finish-nginx-logs-import.png?fit=max&auto=format&n=OwB6o9ddvLojEP8N&q=85&s=e2fe147e4f44cb9485ceed92d7c7310f" alt="Finish Import" width="3812" height="1906" data-path="images/clickstack/finish-nginx-logs-import.png" />
  </Step>

  <Step>
    <h4 id="created-dashboard">
      The dashboard will be created with all visualizations pre-configured
    </h4>

    <Note>
      For the demo dataset, set the time range to **2025-10-20 11:00:00 - 2025-10-21 11:00:00 (UTC)** (adjust based on your local timezone). The imported dashboard won't have a time range specified by default.
    </Note>

    <Image img="https://mintcdn.com/private-7c7dfe99-mintlify-8a08bda2/OwB6o9ddvLojEP8N/images/clickstack/nginx-logs-dashboard.png?fit=max&auto=format&n=OwB6o9ddvLojEP8N&q=85&s=4cd3fd1a810cc70b25e4f1e33ae96c5c" alt="Example Dashboard" width="3812" height="1906" data-path="images/clickstack/nginx-logs-dashboard.png" />
  </Step>
</Steps>

<h2 id="troubleshooting">
  Troubleshooting
</h2>

<h3 id="troubleshooting-not-loading">
  Custom config not loading
</h3>

* Verify the environment variable CUSTOM\_OTELCOL\_CONFIG\_FILE is set correctly

```bash theme={null}
docker exec <container-name> printenv CUSTOM_OTELCOL_CONFIG_FILE
```

* Check that the custom config file is mounted at /etc/otelcol-contrib/custom.config.yaml

```bash theme={null}
docker exec <container-name> ls -lh /etc/otelcol-contrib/custom.config.yaml
```

* View the custom config content to verify it's readable

```bash theme={null}
docker exec <container-name> cat /etc/otelcol-contrib/custom.config.yaml
```

<h3 id="no-logs">
  No logs appearing in HyperDX
</h3>

* Ensure nginx is writing JSON logs

```bash theme={null}
tail -f /var/log/nginx/access.log
```

* Check the collector can read the logs

```bash theme={null}
docker exec `<container>` cat /var/log/nginx/access.log
```

* Verify the effective config includes your filelog receiver

```bash theme={null}
docker exec `<container>` cat /etc/otel/supervisor-data/effective.yaml | grep filelog
```

* Check for errors in the collector logs

```bash theme={null}
docker exec `<container>` cat /etc/otel/supervisor-data/agent.log
```

<h2 id="next-steps">
  Next steps
</h2>

* Set up [alerts](/clickstack/features/alerts) for critical metrics (error rates, latency thresholds)
* Create additional [dashboards](/clickstack/features/dashboards/overview) for specific use cases (API monitoring, security events)

<h2 id="going-to-production">
  Going to production
</h2>

This guide extends ClickStack's built-in OpenTelemetry Collector for quick setup. For production deployments, we recommend running your own OTel Collector and sending data to ClickStack's OTLP endpoint. See [Sending OpenTelemetry data](/clickstack/ingesting-data/opentelemetry) for production configuration.