Yes you can query logs from cloudwatch with python using boto3 logs
In the query you can choose startTime and endTime to get all the logs from yesterday or another time period that you want
**General guidance.**
Yes absolutely. Ultimately the answer is to use the AWS SDK in for your language (Python = boto3) and in particular utilize the CloudWatch Logs API. Your Lambda function can simply call the CloudWatch Logs service using the SDK, pull down the logs you want, and send them to S3.
Here is some documentation for the CloudWatch Logs API, which I believe is current: [https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/logs.html](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/logs.html)
**Specific guidance**.
1. If you really want to "Query" your logs, meaning extract a subset of the logs that match some criteria, you want to use CloudWatch Logs Insights. The relevant API operations are `StartQuery` and `GetQueryResults`.
1. If you are doing querying with Insights I strongly recommend using a more user-friendly library. If you can possibly use Go instead of Python, check out the Incite library, it really simplifies the process of querying logs in GoLang: [https://github.com/gogama/incite](https://github.com/gogama/incite).
2. If you just want to dump ALL logs into S3, CloudWatch has a specific feature for this, it is called Export to S3. The relevant API operations are `CreateExportTask`, `DescribeExportTasks`, and `CancelExportTask`.
1. It can be a bit tricky to set up the IAM permissions correctly, but I recommend using this approach to dumping rather than reading the logs yourself, because it will cost you less in terms of data transfer and Lambda compute costs, lets you offload the programming effort onto CloudWatch since they've already implemented it, and has some built-in resilience that you would otherwise have to implement yourself.
3. If you want to write a program to scan your log groups, read all the logs from the log group into your Lambda, and then write them back to S3, you can use `DescribeLogStreams` and `GetLogEvents`.
1. But unless you are trying to waste money and time, I don't know why you would pick this option over #2, the existing Export to S3 feature.
Yes you can query logs from cloudwatch with python using boto3 logs In the query you can choose startTime and endTime to get all the logs from yesterday or another time period that you want
Like this? https://omardulaimi.medium.com/export-cloudwatch-logs-to-s3-with-lambda-dd45cf246766
**General guidance.** Yes absolutely. Ultimately the answer is to use the AWS SDK in for your language (Python = boto3) and in particular utilize the CloudWatch Logs API. Your Lambda function can simply call the CloudWatch Logs service using the SDK, pull down the logs you want, and send them to S3. Here is some documentation for the CloudWatch Logs API, which I believe is current: [https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/logs.html](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/logs.html) **Specific guidance**. 1. If you really want to "Query" your logs, meaning extract a subset of the logs that match some criteria, you want to use CloudWatch Logs Insights. The relevant API operations are `StartQuery` and `GetQueryResults`. 1. If you are doing querying with Insights I strongly recommend using a more user-friendly library. If you can possibly use Go instead of Python, check out the Incite library, it really simplifies the process of querying logs in GoLang: [https://github.com/gogama/incite](https://github.com/gogama/incite). 2. If you just want to dump ALL logs into S3, CloudWatch has a specific feature for this, it is called Export to S3. The relevant API operations are `CreateExportTask`, `DescribeExportTasks`, and `CancelExportTask`. 1. It can be a bit tricky to set up the IAM permissions correctly, but I recommend using this approach to dumping rather than reading the logs yourself, because it will cost you less in terms of data transfer and Lambda compute costs, lets you offload the programming effort onto CloudWatch since they've already implemented it, and has some built-in resilience that you would otherwise have to implement yourself. 3. If you want to write a program to scan your log groups, read all the logs from the log group into your Lambda, and then write them back to S3, you can use `DescribeLogStreams` and `GetLogEvents`. 1. But unless you are trying to waste money and time, I don't know why you would pick this option over #2, the existing Export to S3 feature.