Skip to content

lambda-promtail - SQS source seems to be not working #15

@Rohlik

Description

@Rohlik

Describe the bug
I created a Lambda that uses Promtail's image to get logs from SQS and push them to the Loki endpoint.
We have a feeling that SQS should be supported based on the source code:
https://github.com/grafana/loki/blob/ffc61fbbf4990ccb00aaed107327fd5ff4d66e82/tools/lambda-promtail/lambda-promtail/main.go#L169
But we are getting the error below from Lambda:

level=error caller=main.go:236 err="error processing event: event bridge event type not supported"

Our pipeline looks like: Log stream > EventBridge > SQS < Lambda with Promtail > Loki
Good to mention is that if we replace SQS with CloudWatch, then it works like a charm.

To Reproduce
Steps to reproduce the behavior:

  1. Create SQS queue
  2. Create Lambda based on the Promtail ECR image and configure it to send logs to a Loki endpoint
  3. Set Lambda trigger against SQS created in step 1
  4. Send some logs to SQS via EventBridge
  5. Errors will appear and nothing will be sent to the Loki

Expected behavior
Logs are correctly processed by Lambda/Promtail and stored in Loki.

Environment:

  • Infrastructure: Loki on EKS, AWS (Lambda, SQS, EventBridge)
  • Deployment tool: Helm, FluxCD

Screenshots, Promtail config, or terminal output
image

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions