jmdha / p6

Query estimation for SQL queries

Projects

Applications

Sunbird - A UCI chess engine using Negamax search in C++

motsog - A UCI chess engine using Negamax search in C

sudokubrute - A Sudoku solver

seeker - A PDDL planner

shapeon - A time series preprocessor

Tools

labyr - A PDDL benchmarker

perfbench - A chess perft benchmarker

lichess data download - Downloads, filters, and extracts chess games from Lichess.org

Libraries

pddllib - A PDDL state and move generation library

jset - A single-header set implementation in C

Parsers

pddlp - A PDDL parser

Misc

.dotfiles

p6's Issues

Integrate the Optimiser into the test suite

Move the CSV logging into a seperate file

Progress bar for Histogram Generation

Make the database systems run parralel test runners

This may cause some issues with console debugging.

Write initial introduction

• Gather what other reports have done
• Sources
• Write what we will do

Make so that we can set what schema a `SuiteData` runs on

Generate output reports from the query analysis

Put database strings in a User Secret file instead of hardcoded

Resize the window to fit progress bars

Missing Condition Attribute from join parser

The variable ConditionTables in JoinNode.cs states the table, but there isn't anything stating the attribute.

Remove all the `QueryGenerator` constructs

Make query parser for MySQL

Officially Handle NULL values (Completely omitted in histograms for now)

Make tests for the `DatabaseConnector` project

Select what schema to run tests on

Make propper errors for missing Secrets

Report Summary

Move the project ReadMe into the front page readme

Get JOB to run

Implement new parser to handle JOB benchmarks

Make tests for the `QueryParser` project

Look into why all the "Larger than" queries right now pretty much all give 0

Implement method to choose best query order

Analyse what is needed for running these #3 tests

• Toolchain?
• What system to integrate with?
• Platform?
• Server?
• etc.

Add theory for histograms

Might include:

• Complexity of making the histogram
• Complexity of using it
• Margin of error as a function of number of buckets or size of buckets
• Difference it makes to have number of buckets be a function of table size, or if it's constant or whatever

Move basic connection properties into the secrets file again

Split up connectors in the codebase

Request a room for Supervisor meeting

Implement a method to set what order the test should run in

Write down potential solution ideas that we can experiment with

Implement the initial table histogram methods

Move the solution file out a folder level in Git

Fabricate test set

• Make what tables and database design we could use (Not SQL, just in concept, maybe drawing)
• What queries should we use?

Make tests for the `Histograms` project

Set target database systems in Secrets file

Move all the database connection into a seperate class library

Fix Introduction

Implement method to parse basic queries for analysis and reorder

Implement method to generate Histograms for each test

Make it an option to auto-start servers or not

TestSet RandomNumberBlocks is dubious

Regenerating the random numbers, each time the benchmark is run, adds uncertainty when later comparing benchmarks

Fix the `About` page on github

Update the CSV writing to include new data from tests

Implement so that the Optimiser can handle the `AND` and `OR` expressions

Integrate MySQL connection into the test suite

Warnings in code

In the bottom of every pull request, github warns:
Unchanged files with check annotations

And lists a bunch of possible null-refs in test-files

Potential optimization: Allow buckets to be larger to contain "the rest" of the last instance.

For instance, in the IMDB data, COUNT(keyword.id=335) is 72496

Nothing is gained from splitting this into many buckets, may as well just let óne bucket contain them all.

Potential solution: When a bucket is made, when it's full, look at the top value, anything following if it's equal.

Make tests for the `QueryPlanParser` project

Find Benchmark

Also find out if we need to run this on a server

Not possible to parse single join statement

Parser can not parse queries like this
SELECT * FROM (A JOIN B ON A.ID = B.ID);