Jun 27, 2022
Thx. There are two options:
1. Filebeat -> Logstash -> both S3 and Elasticsearch - but files on S3 will be JSONs
2. Filebeat -> Kafka -> Kafka Connect -> S3 - files will be Parquets (better option for Athena)
Splitting stream on Logstash/Kafka is a popular way to backup logs/events in colder/cheaper storage.