01 October 2021
Splunk and Fluentd
The Options
Splunk is a commercial product with a limited free use tier. Fluentd/Fluent-Bit and TimeScaleDB are both Open Source projects that should fit together to make a great stack.
The other articles in this series:
ELK Elastic + Logstash + Kibana
Con
No longer an Open Source Solution
ELK was previously available under the Apache license, but new versions are only on a proprietary license. While the new license may still permit more free use than Splunk, ELK can no longer be considered an Open Source solution.
The last time I looked seriously at this product was years ago, before the licensing changes. At that time I concluded that Splunk was a much better solution for the environment and worth paying extra for the licensing.
The core of the ELK stack, Elastic Search is a search engine,built on Apache’s Lucene Core, modern versions of Apache’s SOLR are very similar in capability and how data is queried, and as Apache projects they will always remain under Apache licensing.
Splunk
Pro
Splunk is well supported by the vendor and established in its market role.
- It is a complete multi-platform solution that can handle just about any log type data.
- It scales to massive environments.
- It provides a graphical front end that makes it easy to begin searching.
- It provides tools for graphing, monitoring, and alerting.
Con
- Not Free. Splunk is commercial software, limitations are imposed on the free version that make the free version suitable only for small environments.
- When license limits are exceeded Splunk locks you out of your data for 30 days after you stop the violations.
- The configuration is gnarly.
- Uses a proprietary query language, which creates a steep learning curve for advancing beyond basic queries.
If a temporary situation such as an attack on your website or a mis-configuration issue cause splunk to lock out, support can remove the lock for paid versions. Splunk installs with a trial license, if you don’t convert it to the free license before it expires, you’ll find yourself locked out of your data and needing to purchase a license or wait for 30 days after switching to the free license to get back in.
The configuration structure follows the Java paradigm of having as many configuration files as possible. Splunk also places non-configuration such as installed Apps code into the configuration directory structure. Changes made in the GUI can be written anywhere.
Splunk’s Query Language while suited for the product is very different from SQL, for administrators who only spend a limited amount of time with the product it will take much more time to develop queries than if the language had been based on SQL.
Fluentd, FluentBit, and TimeScaleDB
Pro
- Fluentd supports a wide range of storage back ends including Postgres, MariaDB, Mongo, Splunk, ElasticSearch and SOLR.
- TimeScaleDB is a modified Postgres designed for timescale data. The Query Language is Postgres, the performance is similar on large time series datasets to what Splunk and NoSQL databases offer.
- The Query Language is SQL.
- Fluentd/FluentBit and TimeScaleDB are available in Open Source and Paid versions.
Con
- A lot of Fluent’s plugins are abandoned, including the Postgres plugin!
- You will need to configure everything by hand, unlike Splunk which installs to a state which is at least workable enough to begin experimenting with.
- You’ll need to use a SQL tool to access your data and develop queries. To create dashboards and visualizations you’ll need to use a charting tool like Grafana or create your site with a library like ChartJS. This compares very poorly with Splunk’s Graphical interface. Plus Grafana is under AGPL, which I consider a commercial license.
- You will likely use FluentBit on Forwarders, Fluentd on Indexers, which is two different products/ configurations.
Overall Fluentd will require more work, and have an inferior end experience, with the one big advantage being that you don’t need to learn a specialized language to query the data.