Coder Social home page Coder Social logo

ms-painter / folden Goto Github PK

View Code? Open in Web Editor NEW
19.0 2.0 3.0 286 KB

Cross platform PC application. System wide file watchers handling made simple.

License: MIT License

Rust 100.00%
windows linux rust file-watchers file-watcher windows-service linux-service

folden's People

Contributors

ms-painter avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

folden's Issues

Handler threads thresholds

How many handlers are allowed to be up?

Should there be a limit to how many can be registered?

Configurable by user? Adjusted at runtime by server side main thread?

Log/tail management and publication

Seperate to the workflows.
Any running handler currently prints to console progression or issues.
These messages should be instead handled in a way which could be interwoven into logging and possible runtime tailing.

Each mapped handler to dir can be modified to send output to a known exporter such as a db or log file.

Runtime tailing of a handler's traces/logs should be integrated with the previous part and in some way grpc server to client streaming using tonic.

Extended Foldend logging

Allow methods of supported logging to be chosen between:

  • No logging
  • Env logger (Windows only) logs visible in EventViewer
  • Rolling file logs
  • Database support

Server as a service feature - Mapping serialization

Currently the server mapping data can't be serialized.

This is because of the thread sync sender object stored for each handler,

Which allows us to send messages to handler threads.

We'd want to de/serialize the mapping object to allow starting up handlers already registered on service startup.

Can we store a different object to use to access the sender object through?

Can the sender object on the mapping object be omitted from de/serialization?
Since it would be dead anyways and has no use for the starting up service as it would re-create threads and update the mapping anyways.
Alternatively implementing a de/serialization for this field on the mapping which would store a dead fake sender object.

Schedule based folden handlers

Option between event based and schedule based.

Schedule based would run like startup phase on event based. On each file in directory based on ruleset.
Schedule could be provided like cron based formatting (* * * * *).

TLS allow remote interactions

Wrap grpc inter process server with tls and provide way for clients to supply or store tls certified servers allowed to connect to

Event type specifying per event kind

Events are categorized with sub types such as
CreateKind(Any), CreateKind(File), CreateKind(Other),
ModifyKind(Any), ...

Allow to decide on which of these to accept at [event] workflow configuration for each type of accepted event:

[event]
handle_create = ["File"]
handle_modify = ["Any"]
naming_regex_match = ".*"

Proposal - Publish action/s

Generic action?

Publish action/s for different destinations; Ex.:

  • RabbitMq
  • MongoDB
  • MySQL
  • Splunk
  • Redis
  • Email

Client subcommand logic abstraction

Wrap creation, iteration and execution of subcommand in the client CLI application with an abstraction.

Instead of doing tons of if statements should just iterate over each subcommand supplied to the abstraction object -
Executing based on the inner for's if statement.

Handler trait template mthod implementation

Add Handler trait which template method dictates from Watcher trait what logic should the Handler trait do at runtime.
Each handler provides implementation for Handler trait and Watcher trait implements all needed details.

Startup functionality

Client & Service side:

  • Command to configure if to start handler (or all) on service startup

Handler side:

  • Clarify config option of apply on startup as to attempt to apply workflow on existing files in dir which are not part of the events.

Project logo

Project logo to add to repository and as specialized image files for msi installer

Publisher - http request

Publisher destination send an http request to a set destination.
Supply details in body or url

Workflows overhaul

Currently main idea is to apply one handler to a directory.

Handlers aren't idempotent or atomic currently.

New overhaul describes a way to apply multiple handlers of different order types that could fit in a structured workflow.

What if you could move to a folder then zip the resulting folder?
And what if you wanted after the zip to notify a remote target of this change via a db? Or multiple dbs?

This leads us to the this workflow structure idea:
Folden Workflow Diagram

*In the above Example Arrived relates to data (file/folder) which was just added which file watcher noticed.
**In the above Example Results relates to data results (file/folder) from previous action.
*** In the above Example Filepath relates to file path of the file (results or arrived).

Above all this is the configuration to save tracing which is relevant depending on what you want for the workflow.
So these tracings could not be stored at all or stored only locally or log rotated locally or provided as the data for tracing exported to a target of your choice from message queues to databases.

These workflows have the constraints of being single directional and cannot go backwards to potentially loop parts of the workflow.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.