Current usability for AWS lambda?

[hwine]

Similar to #1, but with a non-app-framework in mind. In particular the logging functionality, but possibly others such as config.

I'm starting a new project intended to be deployed as an AWS Lambda function. Would your recommendation be to start with python-dockerflow, or stick with mozlog for now?

[jezdez]

@hwine That's a good question, as dockerflow is by design targeted at Docker based deployment workflows I'm not sure if that should be used for AWS Lambda deployments. It'd be best to ask the authors of Dockerflow @mostlygeek (and maybe @whd, @robotblake, @jasonthomas) what they think about it.

[willkg]

I just did a AWS Lambda function. I didn't use this package at all. The configuration in python-dockerflow is Django-specific, so that won't fly for Lambda (unless you're doing crazy things).

The logging bits might be helpful, but logs for AWS Lambda go to Cloudwatch, so for it to be really helpful, we'd have to have something that slurps from Cloudwatch and into our logging infrastructure.

Also, if you're using Datadog, then you print the statsd data to stdout and Datadog picks it up by reading Cloudwatch. You can't have that in mozlog format.

I have some rough infrastructure in Pigeon for configuration, logging, and statsd things if that helps. I'd be game for pulling that into python-dockerflow if that helps:

https://github.com/mozilla/socorro-pigeon/blob/735c183ecc626f1bc31648cc68f8f33870aea6c2/pigeon.py#L34

https://github.com/mozilla/socorro-pigeon/blob/735c183ecc626f1bc31648cc68f8f33870aea6c2/pigeon.py#L93

https://github.com/mozilla/socorro-pigeon/blob/735c183ecc626f1bc31648cc68f8f33870aea6c2/pigeon.py#L153

[mostlygeek]

The only time I would consider dockerflow for AWS lambda is if you could spin up docker containers in lambda. Definitely use mozlog for logging as all the downstream log processing is expecting it.

[willkg]

@mostlygeek Are you saying that logging 2.0 pulls logging events from Cloudwatch? I'm pretty sure Pigeon logs aren't going to the logging infrastructure and I was under the impression that wasn't a thing.

[mostlygeek]

@willkg @relud is best to answer that question.

lambda logs go to cloudwatch logs. I don't see why those can't make it into the loggion2.0 pipeline. I don't think this is a currently supported by logging2.0 yet.

[hwine]

I'm new to this group -- would there be value in having a Mozilla (which has MPL) setup for "all the things". If so, adding @willkg logging and other support for Lambda might be worthwhile.

[jvehent]

I'm pretty sure Pigeon logs aren't going to the logging infrastructure and I was under the impression that wasn't a thing.

I most certainly should be. All applications logs should eventually go into the logging pipeline. As for the timeline of doing so, that's a question for ops.

[willkg]

@jvehent @mostlygeek: I had no idea. Good to know. I updated Pigeon to use python-dockerflow for mozlog-style logging.