Pure Storage advanced Monitoring with Splunk: Indexing is Power - Part 1 - Installing Splunk
[ NOTE: machine translation with the help of DeepL translator without additional proofreading and spell checking ]
Splunk is one of the most powerful logging, monitoring and reporting tools on the market. The software can process all logs and metrics from apps, servers and all imaginable IP components and store them in an indexing repository. Graphs, reports, tickets can be generated from the collected data. In addition, stored scripts and workflows/interactions can also be triggered. Another use case would be the facilitated troubleshooting, also here Splunk can be a relief for administrators. Through the previously performed monitor data collection, one can generate consolidated reports of the affected systems at the time of the error, thus connections, interactions and events can be understood at a glance.
Monitoring is already child's play with Pure1. Out-of-the-box ready to use. Proactive storage monitoring accessible free of charge through a cloud portal via web browser or mobile app (iPhone, Android). Pure1 is more than just storage monitoring: analysis down to VM level, capacity planning and performance billing ... just a few of the functions you can use here.
What Pure1 can't do, however: Monitor systems outside of its relevant infrastructure. So e.g. switches, gateways or even applications up to the application layer.
But: this article is not about Pure1. Splunk, as already stated above, can create consolidated reports from a wide variety of systems. In today's Big Data & Analytics era, consolidated data collection and processing is inevitable.
Pure Storage can not only be monitored with Splunk ... Pure Storage can also be your storage/repository for your Splunk instances. Splunk needs to process/index data fast and of course needs appropriate base for that.
At the beginning I want to say: I am definitely not a Splunk expert! Splunk is one of the most powerful tools I had the chance to get to know in my still quite young IT time. Sure, if you only do simple queries - the classic monitoring - you will find your way around relatively quickly. However, the functionality of Splunk would then be felt to 10% actually used.
If you are interested in really deep expertise on the subject, I definitely have the right people to talk to here.
The Splunk installation
Splunk is installed on Linux operating systems 80% of the time. However, I decided to deploy the whole thing on a Windows Server 2016 in English. I allocated 4vCPUs, 8GB RAM and a 60GB virtual disk for the operating system to the virtual machine. For Splunk itself, I created another virtual disk with 100GB. In the course of my research for this blog series, I was told that productive Splunk environments with large amounts of data to process would be very CPU intensive, which is logical ... high-performance data mining simply needs power. For such scenarios, expert knowledge is definitely needed.
Splunk offers the Splunk Enterprise Server in the latest version 7.3 for 60 days completely free of charge for download, only a registration at Splunk is required. After the 60 days have expired, there are two options:
1. the purchase of a license of Splunk Enterprise and
2. "switching" to the free version.
The free version allows a daily log volume of up to 500 MB. The license size is generally calculated based on the daily incoming log volume. In addition to a limited license volume of 500 MB within the free version, other Enterprise features such as user management or cluster functions are disabled.
In the course of the blog series I downloaded the Pure Storage Apps for FlashArray from Splunkbase. More details in blog series part 2. The Splunkbase can be reached here.
The default next-next-finish setup is actually relatively unspectacular, but sufficient for testing purposes. I have only adjusted the installation path here, so Splunk is now installed on the specially created vDisk1 with 100GB on E:/.
After successful installation, you can log in directly to the Splunk WebGUI with the created administration data.
The WebGUI can be reached under http:\HOSTNAME(localhost):8000.
That's it for the setup and the basic Splunk installation.
The next part of the blog will be about integrating a Pure Storage FlashBlade into Splunk.
More info - Links
All officially published setting options in the GUI but also CLI can be read via the "on-board" user guides of the Pure Storage systems.
Click on "Help" in the Purity main menu.
The User Guide is structured like the main menu and can be opened downwards. A search function is also integrated - within here you can also search for keywords.
WEB: Pure Storage (Pure1) support portal - Ticket system and support *(requires registered FlashSystems)
PHONE: Pure Storage phone support: GER - (+49) (0)800 7239467; INTERNATIONAL - (+1) 650 7294088
WEB: Pure Storage OFFICIAL blog