Show HN: Flashlight – A portable tool to shine a light on your data

[ilovetux]

Hello, I am Cliff and I built Flashlight, a versatile data platform.

Flashlight is meant to ingest arbitrary text-based data and allow filtering, transformation and visualization. Think logs, JSON/XML files, JSON REST responses and more.

This is the Public Alpha, I invite everyone to try it out and tell me what you think.

I expect to hear a lot about the lack of documentation. There is currently documentation, but it is not as comprehensive as I think is needed, so I am looking for pain-points and other issues that really NEED to be in the documentation. After that, I will be writing some tutorials to show off a lot of the features that might hide in plain sight.

You can reach me by commenting here or by opening an Issue in the Github Repo.

[ilovetux]

I thought I would provide more information here, as I left it pretty blank when posting:

# Overview

Flashlight is a versatile and powerful platform for ingesting, transforming, and searching through structured, unstructured, and semi-structured data. It allows for interactive searches, dashboards, alerts, and more. Key Features

    Ingest data from various sources
    Transform and normalize data
    Powerful search capabilities
    Interactive dashboards and visualizations
    Alerts and notifications
    Extensible and customizable

# Use Cases

    Log analysis and monitoring
    Data exploration and visualization
    Real-time alerting and notifications
    API data crawling and analysis
    Troubleshooting and debugging

# Project Goals

    Provide a user-friendly interface for data exploration
    Enable quick and efficient data ingestion and transformation
    Offer powerful search and filtering capabilities
    Support interactive and customizable dashboards
    Facilitate real-time alerting and notifications

[kingsizebk]

This sounds awesome, I am going to try it out… It seems like it has features like other similar well known tools (e.g. Splunk, DataDog, etc.) Is that what you are going for?

[ilovetux]

That's a good portion of what we are going for. I want it to be as simple as possible for users to make use of their data.

Whether the data is coming from logs, APIs, files or any other source, it should be simple and straightforward to onboard, search, correlate and visualize.

Once, the data is in Flashlight and you have crafted a couple of searches, then it is easy to put that search into a panel of a dashboard and share the dashboard with coworkers.

So, in essence very similar to both Splunk and Datadog, but there are some important distinctions as well.

* Flashlight will be much more affordable.

* Flashlight is self-hosted, so your data stays safe. Built-in and custom permissions also make it easy to share information with only those people you want to have access.

* Flashlight allows you to gradually model your data. Events are a nice, generic text-based data model, but once your data has special requirements, you can break out that data into a seperate model, which will allow you to specify only the specific fields you need, store the data in its own database table, apply custom permissions, generate CRUD API for the model and much more.

* Search commands, in Flashlight, are meant to be a lot more versatile and extensible. For instance, I have a custom Flashlight search command to connect to a group of servers, pull backups and store those backups for later. There is a corresponding custom search command to restore a specific backup later, so I was able to use those search commands in a "System Backup" dashboard with buttons to retrieve or restore backups.

Please let me know if you are having any issues with Flashlight by opening an Issue in the Github Repo.