temikfart / sql2cypher

Converter from SQL to Cypher.

Hi, I'm Artyom

---

I am a first-year student at MIPT. I have been programming for three years and I try something new all the time. Managed to gain experience in system programming in Huawei and ISP RAS. I am also learning Web Backend Development while leading a team, which is developing StudOko Web application in Kotlin for ATP department at MIPT.

Currently developing several projects and working in ISP RAS over a static analyzer Svace by Samsung.

My CV is here.

Contact me on social media

---

LinkedIn | Telegram | Email

More stuff about me

What I'm currently learning

---

Now, I study Data Flow Analysis (DFA) at work and the Rust language in my free time.

Projects

---

• [C++] sql2cypher is a unique tool, which allows you to translate your queries when migrating from RDBMS (Microsoft SQL Server) into GDBMS (Neo4j).

• [C++] logger is a very lightweight, flexible, simple and universal logger for C++ projects. It supports different formats (in e.g. json), colors, can write logs in some places (files, stdout) and so on.

• [Kotlin] StudOko is a web application that will allow the Department of ATP at the MIPT to issue and accept homework for students, control access to a server cluster and department's GitLab.

Skills

---

• Languages: Java, C/C++, Kotlin and a little bit Python, Ruby, Go
• DBMS: Microsoft SQL Server, Neo4j
• DevOps: Linux, Git and a little bit Docker, CI/CD
• Agile: Scrum & Kanban in Jira

See more in my CV.

sql2cypher's Issues

Translate SQL scheme to Neo4j scheme without queries generation

Now, we translate SQL DDL queries into CQL queries, which creates new dummy nodes and relationships, but it is not correctly. We need to translate it to the Neo4j schema and as a result convert it to some json file, for example.

Add unit-testing for Parser module

Add unit-testing for the enums, classifiers in Parser module. Also, we can test validation methods and methods with the return value.

Translate SQL DML queries to Cypher queries using Neo4j schema in json

Idea: If user have already translated the SQL DDL schema, he has a Neo4j schema in JSON format. We can use this schema to transalte now SQL DML queries and finally get Cypher queries.

Create CMake scripts for packaging the tool

To easy tool installation we need to have packaging scripts.

• For Windows and Linux (.deb) and MacOS packages we can use CPack.

Code Style with Clang Format

To fix C++ Code Style in the project need to use, for example, Clang Format in CI and check it for each merge request.

Refactor app configuration

Now, the configuration module is very bad, because there are a lot of methods for validation with dirty error messages, bad argument parsing implementation, etc.

Add MacOS and Windows builds to CI

Now build.yml runs GitHub Actions only for ubuntu. Need to add build and test scripts for MacOS and Windows for cross-platform testing.

Separate the logger

Now, module, which is responsible for the logging in the application, is very bad coded. I suggest to use open-source logger. Also, we need to remove any logs from configuration module and break the connections between configuration and logging modules.

Cache Action for long time steps in CI

We can use GitHub Actions (cache) for caching steps in CI. For example step with downloading gcc-13 can take 7 minutes.

Extract Lexem class from INode

Now, INode class plays a double role: INode and Lexem. Lexer must parse input file to lexem, Parser must take Array of Tokens/Lexems and create AST nodes. These tokens should have only one DataType and do not know about Statements.

New argument parsing

Need to move arguments parsing to another place. Also, I want to see more clear code.

Do not save separators and syntax punctuation in AST

Now, Parser add to AST, for example, ;, which separates SQL queries between each other, but I think it is not necessary. We can link queries to the root without these separators.

Situation now:

Refactor Syntax Analyzer

• Rename to Parser, it is shorter and more convenience;
• Decompose methods;
• Review logic.

Update README

README.md is not actuale for now. There are no information about new argument parser, how configuration works and how to write tests.

Use formatted strings for validation

We have a lot of invocations of the methods validateX, where we pass an argument: msg. msg is consist of strings, line number and other parts, which are concatenated in one large string.

Maybe it is possible to use formatted strings from C++20 or create analogue in string_utils.

Add unit-tests for AST module

Add unit-testing for the utils methods in include/SCC/ast/common/utils.h and each AST node class.

Extra logs in tests

When some function, for example, PrintHelpAndExit(), prints help info in stdout inside the EXPECT_EXIT macro, this function output will be in the final output near the logs from Google Test framework. This invalid behavior, because test output makes it difficult to read test results.

Generate enums using Python script

We can generate enums and its string representations using scripts like this in ChaiVM.

Store actual MS SQL Grammar in Backus-Naur Form

I suggest to store current MS SQL grammar, which is implemented in Parser, using CLion plugin Grammar-Kit.

Refactor Query Assembler

Query Assembler should be refactored and decomposed as well as Parser (#36).

Cmake build with subdirectories

Now, all build logic placed in the only one CMakeLists.txt in the root of project. Need to use CMakeLists.txt for each directory with sources and tests using add_subdirectory.

Add checks that input SQL queries are supported using Grammar in EBNF

If there is some tool that can check that some input (SQL queries) matches the grammar it will be cool feature for the SCC.

Workflow:

• User runs the SCC and pass as agrument path to the SQL queries;
• SCC at first checks that all of the queries are supported and can be converted;
• if any errors -> report about it;
• otherwise -> translation will start.

Separate build using CMAKE_BUILD_TYPE

Using CMAKE_BUILD_TYPE we can customize project build to reduce configuring and build time. It is useful for local development and CI.

Refactor AST and Tokenizer

• Unnecessary getters and setters in ast.h;
• Outdated docs;
• Unnecessary unused methods;
• Class Node contains two entities inside: lexem and AST node. It should be separated to two pieces.

Add unit-tests for Translator module

Add unit-testing for enums, validation, methods with return value and all cypher units like structures and clauses builders.

Option without parameter should be invalid

If I run SCC with the following command:

scc --sql="/path/to/sql_queries.sql" --mode

where I didn't specified any parameter for the --mode, scc will use default value for the --mode, but I think it is incorrect. Need to raise an error in this case.