kdab / knut Goto Github PK

Knut is an automation tool for code transformation using scripts.

License: Other

CMake 1.42% C++ 89.03% QML 1.01% C 8.54%

automation migration qml qt6 scripting

knut's Introduction

Knut

Knut is an automation tool for code transformation using scripts. The main use case is for migration, but it could be used elsewhere.

The scripts are written either in javascript or QML languages, the latter being able to display a user interface to customize some parts of the application. The script API is available in this documentation, and ranges from normal text transformations (find and replace) to getting contextual information based on the file language (using TreeSitter.

Knut can be used via a command line interface or a user interface.

Requirements

Knut is using Language Server Protocol servers for the GUI. It is not used for the scripts, so if you are using Knut with the command line interface, you don't need it.

For C++, if you are using clangd, note that some tests are disabled for clangd version < 13.

Licensing

Knut is © Klarälvdalens Datakonsult AB (KDAB) and is licensed according to the terms of GPL 3.0.

Contact KDAB at [email protected] to inquire about licensing options.

Knut includes different 3rd party software, you can find the list and licenses for them in the 3RDPARTY document.

Building

To build Knut, you will need an up-to-date C++ & Qt toolchain. This includes:

A recent C++ compiler (GCC 11+, Clang 15+, MSVC 19.40+)
CMake 3.15+
An installation of Qt 6
(optional: Ninja build system)

Download Knut

To clone the code, simply run:

git clone https://github.com/KDAB/knut.git && cd knut

If you are a member of KDAB and have access to our private repositories, you may then simply run:

git submodule update --init --recursive

To download all dependencies.

If you are a contributor outside KDAB, you will need to download all submodules in the 3rdparty/ folder, without the private dependencies in the 3rdparty-kdab/ folder.

git submodule update --init --recursive -- 3rdparty/*

Running CMake

After that you can build Knut with CMake via one of the presets. E.g.:

cmake --preset=release
cmake --build --preset=release

Take a look at CMakePreset.json for a list of available presets.

Running Knut

After building with CMake, run the knut binary from the bin folder within your build directory.

e.g.:

./build-release/bin/knut

Documentation

Visit our comprehensive documentation here: https://kdab.github.io/knut/contributing/getting-involved.html

Contributing

For information about contributing to Knut, see the "Contributing guide" in our documentation.

About KDAB

Knut is written and maintained by Klarälvdalens Datakonsult AB (KDAB).

The KDAB Group is the global No.1 software consultancy for Qt, C++ and OpenGL applications across desktop, embedded and mobile platforms.

The KDAB Group provides consulting and mentoring for developing Qt applications from scratch and in porting from all popular and legacy frameworks to Qt. We continue to help develop parts of Qt and are one of the major contributors to the Qt Project. We can give advanced or standard training anywhere around the globe on Qt as well as C++, OpenGL, 3D and more.

Please visit https://www.kdab.com to meet the people who write code like this

Stay up-to-date with KDAB product announcements:

knut's People

Contributors

Stargazers

Watchers

Forkers

leonmattheskdab narnaud montel mgiroday winterz itzurabhi dfaure-kdab smnppkdab dantti mikom mmennuti-kdab

knut's Issues

Create an "installer" from CI

It would be nice to be able to produce a zip with knut and all the libs it's using when creating a release. I don't think we need a real installer with auto-updates...

Windows zip (using windeployqt
Linux ?
MacOS ?

Optimize cpp2doc

As we are re-adding it, it would be nice to optimize it a bit:

run a profiler, maybe there's something that could be optimized
check if the md file timestamp is later than the cpp one, and if so do not update the documentation for this particular C++ file.

See #59 for discussion on the documentation generation.

Reduce LSP dependencies

Extract the LSP helpers in their own file (they could be made free methods, no need for a class):
- TextLocation::fromLsp
- CodeDocument::toPos
- CodeDocument::fromPos
- CodeDocument::toRange
- CodeDocument::toUri
Remove Lsp dependencies in Symbol
- We could reduce the enum to what is supported, see TreeSitterHelper

Rc: Support alignment flags for static (ie labels) during UI conversion

See here: https://learn.microsoft.com/en-us/windows/win32/controls/button-styles

BS_CENTER, BS_BOTTOM...

Add a removeLines method in CppDocument

We sometimes want to remove a line (most likely based on a QueryMatch).

Would be nice to have a removeLines(RangeMark) method, that removes the query as well as all comments attached to it (either // or /* */).
Also remove the comment on the same line.

Something like that:

// This should stay

RemoveThisLine(); // <- only this line

// This should be removed
RemoveThisOtherLine(); // <- this line + the comment above

RemoveAnotherLine(); // Also remove this comment

Remove Knut.Test all together

We don't need that anymore, there's a TestCase directly in Qt using QTest:
https://doc.qt.io/qt-6/qml-qttest-testcase.html

refactor the test-data/tst_*.qml to use QTest
move the static methods from core/TestUtil to test_utils.h (they are not used anymore in the QML files
remove TestUtil and TestCase.qml from core

Add a find in files API to Project

Would be nice to have a way to find something in multiple files.
The API should return a list of document + position of the find items.

Add unit-tests

We may want to use something like silver searcher - we don't want to open all files inside Knut, would be a waste of memory.

Improve search in document

In the GUI, create an API for generic search. Right now we can only search (Ctrl+F) in TextDocument.
It would be good to have the same search line showing up on the different document, with an API to integrate depending on the view:

Search in QtUiDocument
Search in QtTsDocument
Search in the text view of RcDocument
Search in the left tree of RcDocument
Search in the central view of RcDocument (dialog/strings/menus/...)

Add a command line switch to return the current settings

The switch will be --json-settings, it should work when passing a project (so it returns the project settings)
Returns the current json file for the settings.

Handle `QTextEdit` in the `ScriptDialogItem`

ScriptDialogItem supports a log of different base widgets, but QTextEdit and QPlainTextEdit are missing.
Would be nice to have them working, shouldn't be too difficult as we want to focus only on the text property.

Also allow syntax highlighting: add a property objectNameSyntax to set the syntax used ("cpp", "js"...), and add syntaxHighlighting if it's set (don't if not).

Add an example in the ex_script_dialog.qml

Create a JsonDocument/View

Have a way to see/edit a Json document.

Should be fairly easily to do, see how the QtTsDocument has been integrated:

create a JSonDocument (from TextDocument for now)
add the type in core/settings.json
add the type in the Type enum of Document, and the NLOHMANN_JSON_SERIALIZE_ENUM at the bottom
update the createDocument method in project.cpp
update widgetForDocument from mainwindow.cpp

Refactor doc generation

Refactor the way we are generating the documentation. Right now, cpp2doc is called at every compilation, which is a waste of time, specially if there are no changes.

What we would like is:

remove cpp2doc from the ALL target (need explicit call)
only call cpp2doc during a release, to ensure everything is up to date (doesn't prevent from updating the doc manually)

I wonder what would be the best way to handle that, but surely running cpp2doc every time is a waste of resources.

Add "select larger syntax node" and "select smaller syntax node"

In the C++ menu, we have something call Select Block Up... It would be nice to move that to CodeDocument, and implement it with TreeSitter with "select larger syntax node" and "select smaller syntax node".

That would make it way more useful.

Ideas coming from this blog: https://zed.dev/blog/syntax-aware-editing

Add REUSE compliant license and copyright in pre-commit

See how it's done in KDDockWidgets

RcFile: simplify creation of UI files

The current implementation expect to be able to change to something else, but that's not really true...

So go back to the original implementation:

handle frame conversion directly
remove the rc2ui.js file
use pugixml for writing the ui file

Add a find in files panel on the GUI

This is the next step after #21.

The idea is to have a special panel for find in files, with the list of results displayed. Clicking on a result will open the document and select the word.

Note that results won't be updated if a document is changed after the find (for now).

Should be something like we have in VSCode... if we use rg for finding in files, we could display a bit of context.

Add a way to display log in a ScriptDialog

Currently, scripts report their output through logging.
However, the log output doesn't render whilst the script is running.

This could be a good first step to indicate how far along a script is and to show the user that it's indeed doing something.

We could imagine adding a QPlainTextEdit with a particular name LogOutput or something like that. In the ScriptDialogItem, if we find such QPlainTextEdit, we initalize it in readonly log mode, and redirect the logging output to it.
Hint: there's a small syntaxhilighting in GUI that could be used, see LogHighlighter

Better handling of revision/changelog

In fine, we want to use release-please for doing the release. In order to do that.

We are going to use the simple strategy here: https://github.com/googleapis/release-please/blob/main/docs/customizing.md

Add a version.txt file (we can set the version to 0.1.0 for now)
Use this file text to set the version inside CMake file(READ "version.txt" ver)
Use the version number for the Script QML module (in scriptrunner.cpp`

Improve property detection in the logger

Right now we need to call addProperties in ScriptRunner to find out if the API is a property or a method... this is error prone (like forgetting to add the addProperty).

It would be better to do that automatically. One idea would be to do that during the LOG macro, as we have access to all information (and even an instance of the class maybe to get the metaObject).

Another option would be to do that during the compilation, but it may be more complex...

Add some more API to JsonDocument

This is a follow-up of #20.

Add some API to get or set a value from a JsonDocument. There are more likely some code that could be stolen from Settings, as it's doing exactly that (see hasValue/vlaue/setValue method). Ideally, consolidate those methods, so the code is shared between the 2 classes.

Add tests too.

QML: Rename `Scripts` module

The Scripts name is somewhat nondescript.

We should rename this to Knut, or com.kdab.knut before releasing v1.0.
The exact name still needs to be chosen.

Improve the Tree-Sitter inspector

More discussion, quite some changes as we did a 180° turn.

we keep the inspector as a second window
we add a button for the code views to open the inspector (for better discoverability)
improvement on the query editor
- better syntax highlighting (use scm for that)
- load/save queries (using scm extension for that)
- wiggle on error line

RcFile: create unique ID

There could be multiple ID_STATIC in a dialog, which is an issue to find the right translation.
Fix that by incrementing the ID directly when converting the data, so at least it will be consistent dialog after dialog.

Take a look at convertDialog method in rc_convert.cpp

Note: most likely requires to change all the test data verification files for RC...

Create a demo script for `ScriptDialog`

Create an example script with all the different widgets handled by ScriptDialog, showing:

how you set data
how you get data

Use `std::source_location` for the LOG

Since C++20, there's https://en.cppreference.com/w/cpp/utility/source_location

Maybe we can use that for logging instead of copying the name of the method each time.

Fix: put everything from utils/strings.h into the `namespace Utils`

Right now it's in the namespace Core

Add a way to initialize script dialog from command line

Nowadays, we can run or test a script from the command line using –run or –test.

The one missing piece would be to be able to initialize the dialog from the command line, particularly useful for testing.

Script Dialog use widget name to access data from QML, for example a QLineEdit named fileName will be accessed using data.fileName from QML. We could use a similar mechanism to initialize data. I'm thinking of something like that:

knut --test Script.qml --data `{ fileName:"myfilename"; checkBox: true }`

ie passing either a json fileName or a string that can be interpreted as a json file.

That would mean:

adding a new data option in KnutCore
loading the json file or interpreting the string (using nlohman/json
saving that to initialize the script (most likely needs to be saved in ScriptRunner)
create method ScriptDialogItem::initialize(json) to initialize the dialog from a json
in ScriptRunner::runQml if it's a ScriptDialog, pass the json to it

Finally, once everything is done, add a test:

create a small dialog script with some different widgets (lineedit, checkbox, spinbox, combo...) - see #28
create a test method that check the data values are not the ones initialized in the designer
create a test in tst_knut and pass the data so it works fine

Implement incremental parsing for TreeSitter

TreeSitter is able to update the tree based on incremental changes, would be nice to implement that for performances.
It's not critical though...

See CodeDocument::changeContentTreeSitter

CI ToDo List

As mentioned in #1, apart from setting up the initial CI for testing, there's a few more ToDos:

Additional automations:

Release-please
Automatic documentation deployment?

Create a `CppDocument::memberUnderSymbol()`

This should allow to get the member of a class under the symbol:

returns null if it's not a symbol
returns the Symbol (type Field)

To do that, you need to call CppDocument::queryMember in the header file, and then create the symbol based on the querymatch.
Make sure to go back to where the cursor was when the method was called.

Test: don't show script dialog in test mode

When running a test, don't show the script dialog, it's quite disturbing, and we want to click on the buttons :)

Concurrency test missing

Simplify API documentation

This is a follow up of #49

Now that we only have a Script module, it doesn't really make sense to explicitly says it's in this module.

Remove all \inqmlmodule Script in the code documentation
Change cpp2doc tool to always put in the Script module if there are no \inqmlmodule (we want to be able to use it, just in case for later)

Bump minimum Qt version to 6.3

This will allow us to:

Replace QT_GLOBAL_STATIC_WITH_ARGS with QT_GLOBAL_STATIC

Improve nextStep API in ScriptDialog

The nextStep API is good for big independent steps, but bad showing progress of one action (for example, the same action being done on 1000 items).

It would be good to improve the API:

if the string is empty (ie calling nextStep()), then repeat the previous string
don't update more than 5x times per seconds (meaning you need a timer to do the update).

Would be good to have some numbers to see how it impacts script with a before/after.

Macros can break Tree-Sitter parsing

Example:

class AFX_EXT_CLASS CPrjDbase6 : public CDialog
{};

This is parsed entirely incorrectly:

It's a known bug in tree-sitter-cpp, that doesn't seem to have an easy solution.
tree-sitter/tree-sitter-cpp#85

It would be good to be able to fix this from within Knut somehow.

Create a symbols tree view

Would be nice to have a to have a view of the symbols in the current file, mostly for us developer to ensure we have everything right.

Maybe as an external window for now, make sure to:

delete the dialog on close (like the treesitter inspector)
make sure it works even if a non-code document is current

Review public API

For each API, ensure there is:

documentation (property or method)
ensure the documentation is properly rendered
proper LOG

List of API to review:

Migrate QML integration to modern QML

This seems a not so easy tasks:

Use macro like QML_ELEMENT, QML_SINGLETON
Remove code to integrate the singleton
Add necessary CMake Qt macros

But if I understand, it would remove some code, particularly in ScriptRunner, but also we won't have to handle Q_DECLARE_METATYPE...

Integrate QML parser for TreeSitter (most likely requires a 3rd party)
Make QmlDocument a CodeDocument sub-class
=> you should see symbols in Knut Gui
Add some specific API to extract an item with its properties
=> would be nice to easily do some introspection, without using TreeSitter query (they are used internally)

Refactor the text range/mark API

This is related to #54 , we currently have 4 different classes to handle position in text:

simple int: that's the general position in a TextDocument
TextRange: simple classes with start/end position
TextLocation: contains a TextRange + a document
- use only for CodeDocument::references
- shouldn't be documented as it's not even public API
Mark: position in a document, which is updated based on edits
RangeMark: as the name suggest, that's a range of 2 marks, for start/end
- very similar of TextRange and TextLocation, except for the auto update

I would like to reduce to 3 different classes:

simple int: we keep this one for recording script, it's perfect
Cursor: position in a document (ex Mark)
CursorRange: range of cursor (ex RangeMark)

Cursor is a better term and more explicit, and I don't think we need the non-document aware version:

either you want it for reading something, and it's not a problem
or you want to keep it longer after edit, meaning you want updates.

Add ribbon to the RcViewer

Just add an entry for ribbons in the RcViewer, so we know it's there...

Just display the name of the ribbon, no need to show more details yet

Improve current documentation

There is a lot to do to improve the current documentation:

check the install doc now that it's on github
make a tour of the GUI
better document the CLI
explain how to create a non-GUI script
explain how to create a GUI script
explain how to do script testing
dive into the TreeSitter
create a small tutorial