Use database as data source? about nl4dv

[arpitnarechania]

Good point and I agree having that flexibility would be most desirable! Initial thoughts: using a database may not be practical or feasible due to the following reasons:

1. NL4DV currently supports .tsv, .csv, and .json as valid input file formats; these are representations of a tabular dataset with attributes (columns) and records (rows). Supporting Postgres (a relational database powered by SQL queries to perform operations such as JOINs) and Elasticsearch (a document-oriented NoSQL datastore), or even other paradigms such as "streaming data sources" might be out-of-scope for this toolkit, which is currently focused on static tabular datasets.

2. NL4DV generated visualization specification is currently powered by the Vega-Lite grammar. Per their documentation and one of their issues, they too currently support only tabular representations of data. However, I did find this recent discussion on using Vega-Lite in the ELK (Elasticsearch, Logstash, Kibana) stack; maybe it helps you in your use-case.

Having said that, can you share your desired requirement(s) a little more? Reviewing them would help me better understand your needs and accordingly suggest next steps.

[npow]

We have a lot of data indexed in Elasticsearch, and perform visualizations using Kibana. This works, albeit with some initial learning curve to use Kibana, which is not ideal for non tech-savvy people. So I was thinking maybe there's a way to go from natural language queries straight to the visualizations, and came across your library.

Some common queries we perform are:

• viewing aggregations (averages, counts) over time, along with breakdowns by multiple dimensions
• 2d visualizations (bar charts, stacked bar charts, pie charts)

So a user should be able to enter a query like: "count requests over the last 2 weeks, hourly", which would then render a chart with timestamp on the x-axis, and counts on the y-axis (could be in Kibana, or elsewhere).

[arpitnarechania]

Thanks for providing additional information. Having used ES/Kibana extensively myself, I am familiar with its querying engine and visualization capabilities.

Your query, "count requests over the last 2 weeks, hourly" is not based on the data itself (i.e., the attributes and records); instead, it focuses on metadata which ES generates and indexes along with the data, e.g., the ingestion/request times of individual documents/records.

In general, NL4DV is not powerful enough to support relative temporal queries (e.g., last 1 month, yesterday) or even custom grouping by time (e.g., hourly); it is currently strictly usable as absolute date filters, e.g., "show movies released between 2005 and 2013".

Since grouping and filtering relative to different time periods is a key functionality of ES/Kibana, I'm afraid NL4DV is not the ideal tool for your use-case (sorry!) as we don't have active plans on adding support for those features, yet; I will get back to you in case this changes. Meanwhile, I will be more than happy to review if you plan to submit a Pull Request for this functionality.