#sre - November 2019

You might consider using kinesis

Stream the logs first there

Then from there stream them to cloud watch and s3

Erik, thanks for your answer. So, we already use Kinesis for streaming logs to s3 but Kinesis does not have the ability to stream logs from s3
I think we should use some Lambda function for this purpose, the same way as DataDog uses for getting logs from customer's s3 buckets
But anyway thanks a lot

No, you don’t stream from s3

But you can output to both s3 and cloud watch logs at the same time

Which is I think what you wanted to achieve

Yes, that is I want, but looks like kinesis can not stream to CloudWatch

Hrmm

Look into AWS glue?

@Andriy Knysh (Cloud Posse) might know more

He worked on something similar

I want to implement something similar. Stream to s3 and elastic search

Yeah, I saw this manual
https://aws.amazon.com/ru/blogs/security/enabling-serverless-security-analytics-using-aws-waf-full-logs/
But it brings several additional tools and looks like very overloaded

I have found this lambda function https://github.com/miztiik/serverless-s3-to-elasticsearch-ingester
So, pyton code very similar to the lambda function that datadog uses)
I am going to update that codebase and add additional function - streaming to CloudWatch

why do you need to move it from S3 to CloudWatch? What’s the use case?

Hi, Andriy. Nice to see you
We are using AWS ALB + WAF. They can streams their logs only to s3 bucket. Also we are using CloudWatch as one point for monitoring and logging our applications and infrastructure components. So, we need some tools that can get logs from s3 and send them to CloudWatch

I want to replace ELK stack that we use for log collecting in favour of Fluentd + CloudWatch. Now I can collect logs from customer application and kubernetes cluster to CW, also we use CW Container Insights as Kubernetes monitoring tools

hey Sergey, yes looks like lambda is the only way to go to move logs from S3 to CW

Good morning All,
I'm looking for some advice around monitoring and wondered if somebody had the time to give me their opinions.
I work for a company that's got multiple AWS accounts, One company account (master) and then one per client which contain many VPC's normal separated based on DEV, Test, Pre-Live and Prod per product we design. Primarily web sites or applications.
We've historically used Cloudwatch and created the dashboards as an when we need something above and beyond the default EC2 monitoring but this is become costly and doesn't really give us a centralised approach or the flexibility to add something that is beyond the basics.
I'm looking to bring us in to a more centralised way of working and have been reviewing Prometheus + Grafana and maybe Thanos to get a more historical view of data. I've seen the SweetOps articles on Prometheus and read over the kubernetes example but I'm left confused as to what is the best way to architect a Prometheus platform. Most examples I've seen look to deploy Prometheus on kubernetes, but I would have thought it would be better off as an EC2 node. There also appears to be a split in the thinking that its would be best to deliver a Prometheus pair of server to each AWS account rather than trying to bridge from say the Companies Master account. Appreciate this is a wide and very generalized question but at this time any comments would be of help to me.

@Gareth those docs are a bit out of date

Sorry

Join us this coming Wednesday for #office-hours and we can discuss

Hi all, anyone set up webpage client side metrics getting to Prometheus? I wanted to build a poc and I've found this https://www.weave.works/blog/browser-metrics-into-prometheus/.
Does anyone have a different solution that would like to share, just to get some different options

Hey all, I'm looking to use https://github.com/cloudposse/prometheus-to-cloudwatch. Trying to figure out what my cloudwatch_namespace is. Any advice on figuring it out?

My service is running in ECS with a FARGATE deployment

@Andriy Knysh (Cloud Posse) ☝️

@Mitch this might help https://github.com/cloudposse/prometheus-to-cloudwatch#screenshots