vmware-archive / hillview Goto Github PK

In many scenarios (like heavy hitters), it is natural to sort entries by counts. For instance, sorting airlines by number of flights that they operate.

They should be tagged per-client and deleted when the client disconnects.

After we run heavy hitters, we should filter the results by computing the exact frequency counts.

Steps to reproduce:

• Load one of the flight datasets
• Select columns ArrTime and ArrDelay
• Right click on one of the columns -> Heat map
• The trendline should follow the green colour, and it does
• In the heat map, select view -> swap axes
• The trendline should follow the green colour, but it doesn't.

When viewing heavy hitters on a column (or a group of columns), we should display those columns.

Currently, the only way to select rows is to first go to the histogram and then select. We might want to support selection in the table view.

We should use the proper copyright and license templates in all files.

We need a back button!

The context menu that is displayed for a table should depend on the selected columns.

Labels which are too long are drawn overlapping.
When using date-times sometimes labels do not show the date, only the time.

https://github.com/vmware/clarity

Maybe it would be nice to show only one progress bar for a chain of dependent operations?

Currently, the old table disappears after a filtering operation.

For most storage substrates the schema has an inherited column ordering.
Today this is lost.

I commented out some lines in RecordOrder.java (merge.add(outcome) and these tests still produced the same results!

We should tweak the parameters to make them more precise.

Suppose we are viewing a histogram where data is sorted by attribute A. Suppose we click on the table view. Currently, we get the table view where no columns are displayed. Ideally, we should be seeing the data sorted by attribute A (in the same order that the histogram uses, which is presumably ascending.)

When building the front-end (mvn package in web), the build system mentions a missing file:

ERROR in ./rpc.ts
(2,22): error TS6053: File '/Users/hkruiger/Projects/hiero_fresh/web/src/main/webapp/typings/index.d.ts' not found.
   [1] ./rpc.ts 6.61 kB {0} [built] [1 error]
ERROR in ./rpc.ts
(2,22): error TS6053: File '/Users/hkruiger/Projects/hiero_fresh/web/src/main/webapp/typings/index.d.ts' not found.

Removing the violating import line (line 2 in web/src/main/webapp/rpc.ts) removes this error, and I don't see any problems in the application after that. But I'm not sure if this is a good fix, as the file may need to be imported for a reason(?).

After creating a table of Integer columns and splitting it using SplitTable, the columns of the subtables are changed to ObjectArrayColumn. As a result when invoking the column's asDouble() method the default converter from ints to doubles is not invoked. If an explicit converter is not supplied this results in a null exception. Below is a code that generates the bug:

Test
public void createBug() {
// Creating Int Table
final SmallTable bigTable = getIntTable(10000, 1);
// Grabbing the Column
String colName = bigTable.getSchema().getColumnNames().iterator().next();
IColumn column = bigTable.getColumn(colName);
IMembershipSet memset = bigTable.getMembershipSet();
IRowIterator iter = memset.getIterator();
// All seem to work fine
System.out.println(" printing the double " + column.asDouble(iter.getNextRow(), null));
System.out.println(" printing the double " + column.asDouble(iter.getNextRow(), null));

// Splitting the table
List<SmallTable> tabList = SplitTable(bigTable, 10000);
// Grabbing the column from  the sub-tables
ITable subtable = tabList.iterator().next();
IColumn column1 = subtable.getColumn(colName);
IMembershipSet memset1 = subtable.getMembershipSet();
IRowIterator iter1 = memset1.getIterator();

//Null Exception!!!
System.out.println(" printing the double " + column1.asDouble(iter1.getNextRow(), null));
System.out.println(" printing the double " + column1.asDouble(iter1.getNextRow(), null));

}

See branch mbudiu, test DataSetTest.unsubscriptionTest

Netty gives an error saying that we exceed the framesize limit. I think it's because CorrMatrix uses too much memory. I'll look into this, there'll likely be some memory optimizations possible.

Conversion to string does not always show hours, minutes, etc.

The CDF is computed at the bucket resolution, but it should be computed at the pixel resolution.

All datasets are hardwired in the UI.

e.g., intersection, union, etc.
These will trigger exceptions at runtime.

Displaying the DepTime column shows .2% of the data. However, after page-down it jumps to offset 2.7%.
This seems to be related to the handling of missing values.

We should change to a workflow that makes use of these annotations.

web/target/web-1.0-SNAPSHOT.war doesn't exist yet when installing the required software (before building the project). Tomcat should be configured after the front-end has been built.

Using ctrl-mouse in charts should perform a complementary selection.

Must catch exceptions in sketches and maps and handle them ourselves.

rpc.ts has the following line:

const HillviewServiceUrl : string = "ws://localhost:8080";

We need to determine the URL dynamically instead.

CompositeException (which is used internally by the Rx framework) cannot be properly serialized using kryo.

Invoking the same operation on the same dataset should return the saved result if it still exists.