
But that also makes it costly in terms of resources. Logstash can execute plugins which give it a lot of power. Documents (log records) are being sent to Logstash where they can be transformed, enriched, sent to other loggers, etc. Logstash - separate component that sits in front of Elasticsearch.
Direct API call - POST to Elasticsearch directly (usually not what you want).An index is identified by a name (that must be all lowercase) and this name is used to refer to the index when performing indexing, search, update, and delete operations against the documents in it. a log line is a document (structured record)
Document: basically a record, but it doesn’t have to be structured. Spinning up a cluster is out of scope for this post. Basically it’s a good setup for a proof of concept or for starting with Elasticsearch. All of these have their place and advantages, but might not be needed right away. No Logstash, CloudWatch, Kibana Firehose or any other thing like that. It involves an Elasticsearch cluster and a server to send logs from. It’s hardly AWS specific, but it assumes an AWS Elasticsearch cluster and has a few notes regarding that. Possibly the way that requires the least amount of setup (read: effort) while still producing decent results. Nginx Logs to Elasticsearch (in AWS) Using Pipelines and Filebeat (no Logstash)Ī pretty raw post about one of many ways of sending data to Elasticsearch.