distributev / mailmerger

mailmerger

The scope of this project is to build a simple prototype around - https://github.com/cuba-platform/yarg

The code will be written in Java8 using Gradle as a build system.

The project will have 2 sub-folders (sub-projects for gradle)

1. mailmerger-cmdline


2. mailmerger-server

Both folders will follow the standard Maven project folder structure - https://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html

The main functionality will be built in mailmerger-cmdline (as a command line application) while mailmerger-server should only be a light "server" wrapper around the previous functionality from mailmerger-cmdline

Both mailmerger-cmdline and mailmerger-server will be built using standard Gradle multi-project (mailmerger-server will have mailmerger-cmdline defined as dependency)

mailmerger-cmdline

mailmerger-cmdline will contain standard Junit 5 tests to prove the functionality works fine.

mailmerger-cmdline will be a straightforward wrapper around yarg reporting engine and will

• parse an input CSV file containing report data rows
• for each row in the input CSV generate 1 ouput PDF file using yarg - for instance if input CSV has 4 rows this will generate 4 output PDF files, one per input CSV row

You could look at the most basic yarg sample (yarg comes with samples) and build mailmerger-cmdline around that sample.

MailMerge command line usage will be the following

• -c, --c path_to_xml_configuration_file - Optional argument, if not provided a "default" configuration file will be used
• -f, --file path_to_csv_data_file - Required argument with the path to the CSV containing report rows

Libraries / Dependencies

https://github.com/cuba-platform/yarg (Report Generator)
https://commons.apache.org/proper/commons-csv/ (csv parser)
https://freemarker.apache.org (templating)
https://commons.apache.org/proper/commons-cli/ (command line)
https://github.com/google/guava (utilities library - Google core libraries for Java)
https://commons.apache.org/proper/commons-configuration/ ( generic configuration interface)
https://logging.apache.org/log4j/2.x/ (logging library)
https://junit.org/junit5/

mailmerger-server

mailmerger-server will basically take the previous mailmerger-cmdline functionality and "decorate" this functionality as a "server" app which will always "watch" a specified folder for new CSV files to process. Whenever a new CSV file is found in the "poll" folder it will be picked up and processed automatically. That's all. gradle-watch-plugin will be used to "watch" the "poll" folder

Libraries / Dependencies

spring-boot-2 (with groovy as language) - https://start.spring.io
https://gradle.org 
https://github.com/bluepapa32/gradle-watch-plugin (Run predefined tasks whenever watched file patterns are added, changed or deleted.)

Logging

mailmerger-cmdline Log4j2 will generate 2 log files

• logs/info.log
• logs/errors.log

Both files will use "ConversionPattern" value="%d{dd/MM/yyyy HH:mm:ss} [%t] %p %c - %m%n"

XML configuration file

Building and running

Building

To build and run tests

./gradlew build

mailmerger's Issues

remove org.springframework.boot dependency from mailmerger-cmdline

Only mailmerger-server should depend on springboot

From the specification here are the list of mailmerger-cmdline dependencies

https://github.com/cuba-platform/yarg (Report Generator)
https://commons.apache.org/proper/commons-csv/ (csv parser)
https://freemarker.apache.org (templating)
https://commons.apache.org/proper/commons-cli/ (command line)
https://github.com/google/guava (utilities library - Google core libraries for Java)
https://commons.apache.org/proper/commons-configuration/ ( generic configuration interface)
https://logging.apache.org/log4j/2.x/ (logging library)
https://junit.org/junit5/ (test dependency)

I'll provide some details of the logging shortly.

Cannot execute server 2

Then I tried various combinations in the DOS cmd prompt but still not luck - in the above screenshot you can see my folder structure

Please document both

1. How to run Server from Eclipse - detailed steps
2. How to run Server from DOS cmd - so I can copy paste the working command

All junit tests should assert and should validate whatever need to assert / validate for making sure the functionality works fine

If I suddenly give concurrently few hundreds CSV files into the poll folder

Will I "semi-randomly" loose / miss couple of CSV files which will not be processed but will still disappear from the poll folder?

Does gradle -watch documentation says anything about this situation?

From previous experience with similar "folder polling" approaches (different implementations) I know that it can be tricky to get a solution which can guarantee that it will never loose any input file. Further more I noticed that it is more likely to loose small input CSV files (let's say files with only 1 row of data) than it is to loose bigger CSV files (files with 1000 rows) though this is not always a clear rule.

Please double check our code / implementation together with the gradle documentation and let me know what do you think about this.

How to run cmdline?

I tried various combination of arguments and could not get it running. Please let me know the correct arguments I need to pass to get it running. Below you see an example of what it was tried.

gradle dependency tree

In Eclipse with Maven it is possible to view dependency tree graphically like this -

http://www.splessons.com/lesson/maven-dependency-tree/

I'm looking for something similar with gradle Eclipse - Please evaluate if this is a good option and, if so, configure this plugin for our project.

https://github.com/vanniktech/gradle-dependency-graph-generator-plugin

Why I get stacktrace inside info.log?

replace ${Root.value1} with ${value1}

Everywhere where freemarker expressions are used (i.e. in templates or in the xml configuration for file / folder names) allow referring variables with short syntax ${value1} - internally and technically Root. might be required but for normal users ${value1} makes sense - i.e. ${customer}, ${country} ${city} ,etc.

Can I use @grab inside .gradle scripts?

Can I do this inside the gradle scripts? This is a built-in groovy feature, gradle is groovy so I would expect that this is possible. Can you please test?

http://docs.groovy-lang.org/latest/html/documentation/grape.html

Sample of run configuration to execute server from Eclipse?

Related with the issue #12 please give me a screenshot how to run the server from Eclipse - Run Configuration which will consider the relative path folders which will get the command running.

Please show me a screenshot how to run with the currently eclipse configuration / relative folders

Here is the help.md documentation ==> how this should be adapted to get it working in Eclipse with relative folder paths?

You can then run org.distributev.mailmerger.server.Application with the following program arguments

--project.dir=/full/path/to/folder/containing/watch/build.gradle
--watch.dir=/full/path/to/folder/to/watch/for/data/files
--completed.dir=/full/path/to/folder/where/data/files/will/be/placed/when/completed
--config.file=/full/path/to/mailmerger/config/file

errors while importing gradle project into eclipse

I tried to import the gradle project into eclipse and I get lots of "Classpath" errors

Importing should work out of the box without further manual Classpath modifications

gradlew.bat build install fails

Why this error is not reported in errors.log?

Is it possible to write gradle scripts without the buildScript header?