ushnisha / jobshop-minimal Goto Github PK

Currently, all the debug messages are being printed out to the screen; we need better logging capabilities:

(1) send them all to a suitable log file.
(2) provide a new option to specify the log file location; default location is directory from which the program is invoked
(3) allow logging to both screen and file simultaneously
(4) logging mechanism should incorporate the DEBUG level functionality to the extent possible

Add a suitable copyright message. For example, the following message is based on the format used by IBM in its open source projects.

/**

Copyright (c) 2017 Arun Kunchithapatham
All rights reserved. This program and the accompanying materials
are made available under the terms of the GNU AGPL v3
which accompanies this distribution, and is available at
https://www.gnu.org/licenses/agpl-3.0.en.html
Contributors:
Arun Kunchithapatham - Initial Contribution

*/

Update the README.md with details on the project, how to compile, review license and other details.

In the gantt chart, taskplans that are pegged to a late demand must be red in color; taskplans that result from a released work order must be gray in color; other taskplans have the default color.

In the demand view, demands that are late must be in red; demands that are on time or early must be green in color.

Update README to indicate new functionality supported by the tool.

Consider test0010 (constrained planning test) with the following modifications:

PART001 demand is increased to 846 (the effect is to increase the lead times of OP10 and OP20 to 3 days from 1 day with the original demand of 270).

PART001 demand is moved out by one day and is now due on 2018-01-15.

PART002 demand is not modified in any way.

The net result of this is that during simple JIT planning of PART001-OP20 will start it on 2018-01-05 and end on 2018-01-08 so that it can be shipped out to meet the demand on 2018-01-15.

For PART002, simple JIT planning will cause its PART002-OP20 to start on 2018-01-06 and end on 2018-01-07 so that it can be shipped out to meet demand on 2018-01-14.

However, a bug in the current workcenter constraint/overload check does not find this corner case where an earlier planned task (PART001-OP20) completely subsumes the later planned task (PART002-OP20). As a result, we create a workcenter overload (infeasible plan).

Need to add additional code to handle this corner case and add a suitable test plan for the same.

Static analysis can be used by the solver to speed it up or to reduce the search space or to come up with smarter algorithms (directional search) etc. This could include:

(1) Determine the "level" of each task and workcenter (so that we can use that to perform upstream vs. downstream sweeps of certain kinds)

(2) Determine the criticality of workcenters. This will help us solve overload problems on the critical resources first. The meaning of the word critical is still unclear. The solution should also define what criticality means while coming up with ideas for determining criticality of resources.

(3) Determine EPST and LPST of tasks. Since the quantity of the taskplans also determines it length, the EPST and LPST are really dynamic quantities; a taskplan may be split into two smaller pieces and simultaneously assigned to two alternate resources; that would change the EPST/LPST calculations! So, any such information should be a guide to the solver; and we must always remember that it can lead to sub-optimal plans (as with any heuristic planning approach).

(4) Determining if the jobshop can be broken up into independent subproblems that can be solved in parallel.

(5) Anything else?

Some tests plans can have multiple optimal solutions; when the number of optimal solutions is small/easy to enumerate, we should support multiple expect files and check against all of them - even if one matches, then the testplan passes; if none match, the testplan fails.

Currently, output is written to the screen when input_mode is FLATFILE; but it is not written in an easily parsable manner (easier to read for testplan output, but not easy to load into a spreadsheet and filter/pivot etc.).

In addition, the test plan output is captured by redirecting stdout to a file; rather we should introduce a TESTPLAN mode and when run in that mode, the output for testplans must be written to an output file that can then be compared to the expect file. Output to the screen may change, but that should not affect the testplan output data (they should be managed separately).

While doing this, add some additional output files that give us workcenter utilization statistics. The underlying code will be useful when we get around to implementing minimum utilization requirements.

There is a bug in the planning logic that came to fore only when creating testplans for Base Scenario 2 (yet to be published). In this case, when last task (before meeting demand) has multiple workcenters that can be loaded, but in either case, the demand will be delayed (that is the task will be planned later than JIT to meet demand on time), the logic is incorrectly choosing the primary workcenter at all times; that can result in more lateness than necessary if the alternate workcenter has available capacity earlier than the primary workcenter.

Comma separator for the option file makes the option file difficult to read/interpret. Changing the separator to comma.

(1) Change JobShop.java source code
(2) Update run_tests.sh that auto-generates the option file for the test plans
(3) Update the documentation/Wiki appropriately.

Run test0011; the following output is produced for TaskPlans:

TaskPlans:
PART001-OP10 [ 2018-01-02T00:00 - 2018-01-03T00:00] Qty: 270; DemandID: DMD001; Plan: Basic Plan; Loads: null
PART001-OP20 [ 2018-01-03T00:00 - 2018-01-04T00:00] Qty: 270; DemandID: DMD001; Plan: Basic Plan; Loads: RES003
PART001-SHIP [ 2018-01-04T00:00 - 2018-01-11T00:00] Qty: 270; DemandID: DMD001; Plan: Basic Plan; Loads: null
PART002-OP10 [ 2018-01-02T00:00 - 2018-01-03T00:00] Qty: 270; DemandID: DMD002; Plan: Basic Plan; Loads: null
PART002-OP20 [ 2018-01-04T00:00 - 2018-01-05T00:00] Qty: 270; DemandID: DMD002; Plan: Basic Plan; Loads: RES003
PART002-SHIP [ 2018-01-05T00:00 - 2018-01-12T00:00] Qty: 270; DemandID: DMD002; Plan: Basic Plan; Loads: null

Note that the OP10 step for both PART001 and PART002 are not loading any workcenter (shows "null"). That is not correct. We need to load either RES001 or RES002.

Add a new testplan for cycle detection logic.

Add suitable documentation for the test cases. The Wiki format is not suitable for this documentation. So, create a suitable folder and html files within the tests subdirectory to document the different test cases.

Currently, the test plans in the tests directory have the plan name as "Unconstrained" or "Constrained" but this is simply misleading. Whether the plan generated by the solver is constrained or unconstrained depends on the planning parameter and not the plan name.

Best to change the plan name to something very generic for all test plans and let the documentation describe the test case and the nature of the plan/constraints in more detail.

(1) Use the header template provided by GNU

(2) Update the dates to include 2018

Although the data/schema specify demand priority the basic planning algorithm is not planning demands in the order of their priority.

Assume that the lower the value of the priority field, the more important the demand. This will show up in cases of capacity constraints where the lower priority demand will be pushed out and higher priority demand will be planned on time (or at least as early as possible).

The access control modifiers are too permissive -- way too many public classes and methods. For all practical purposes, the end user of this package should not need to have access to any of the classes except the JobShop and Plans classes and very limited functions of these classes too.

• Change all classes/interfaces except JobShop and Plan to default (private-package) access.

• Change as many of the member functions to private as possible and the rest to default (private-package) access for all classes.

• Minimize the assumptions about default option values when creating the JobShop class. This should include:
  -- forcing user to provide an option file (fail fast if valid file does not exist)
  -- forcing the user to select input_mode and depending on the input_mode, either the datadir or db_connection_string options; fail fast if these requirements are not met

• Any other cleanup ideas? Add to the comments.

Test out the current code trying to load from postgresql database. Make and suitable changes to schema, data files and code to allow compatibility to use both sqlite and postgresql. Capture any changes in the comments of this issue (and in the commit message).

(1) Determine the criticality of workcenters. This will help us solve overload problems on the critical resources first. The meaning of the word critical is still unclear. The solution should also define what criticality means while coming up with ideas for determining criticality of resources.

(2) Determine EPST and LPST of tasks. Since the quantity of the taskplans also determines it length, the EPST and LPST are really dynamic quantities; a taskplan may be split into two smaller pieces and simultaneously assigned to two alternate resources; that would change the EPST/LPST calculations! So, any such information should be a guide to the solver; and we must always remember that it can lead to sub-optimal plans (as with any heuristic planning approach).

(3) Anything else?

Minimal data validation is required for flat file loading; with databases, the schema should hopefully enforce most of this.

(1) Find duplicate data (primary key constraint violation and unique constraint violations )
(2) Find missing data (foreign key constraint violation)
(3) Find cycles/loops in structure (could be a long cycle - needs to be done in code for all models as part of static analysis)
(4) Anything else?

The "make clean" commands are not cleaning up all of the javadoc generated files. Make suitable changes to remove all generated files in all generated directories. At the end of a correct "make clean", running "git status" should not show the "docs" directory as having untracked files.

Latest javadoc throws an error when the documentation text has angled brackets. For example, if one of the @param or @return values is described as a List that throws an error.

The fix is to add an additional option to the Makefile (-Xdoclint:none). This issue is a placeholder to capture this problem (even though this is a very trivial change).

The PlanParams class is not necessary. The map of planning parameters can just as easily stored directly on the plan model and we can avoid having to jump through the PlanParams model to get to these values; there is no value added by having a unique the PlanParams model that is anyway keyed to a specific plan.

Suitable changes need to be made to other classes to effect this change. sources/com/ushnisha/JobShop/PlanParms.java will be removed.

Currently, workcenters support only efficiency calendars. We also need to support minimum utilization calendars. This includes:

(1) Any additional data/schema changes to represent minimum utilization calendars
(2) Planning logic to try and satisfy minimum utilization % requirements and the order in which we try to load the resources up to the minimum utilization
(3) Clearly define any assumptions in terms of the priority of a minimum utilization goal when compared to meeting demand on time, etc.

Required for debugging print statements.

In the current logic, if a Task is planned earlier (due to holiday constraints, for example) it is possible for the downstream tasks to be pulled ahead and the demand is satisfied earlier.

Need to add logic to let the users define if it is ok to delivery the demand early or delivery JIT. We also need to decide that if only the final delivery/shipment task must be planned JIT or if all tasks must be planned JIT. That is, where should the inventory be held?!

Need to think about this more. We don't want too many flags/user controls either at this time as it may complicate the solution too early.

Update make files to:

(1) Create any missing empty directories that do not get uploaded to github

When data is uploaded to sqlite databases, we have a postprocessing script that fixes the datetime format to match the jdbc driver formats.

The correct format is: %Y-%m-%d %H:%M:%f

However, the scripts (both sql script used by the shell scrip loader as well as python loader) have an incorrect format: %Y-%m-%d %H:%M:%S.%f

As a result instead of getting: 2018-01-01 08:00:00.000
we instead get: 2018-01-01 08:00:00.00.000 which is incorrect. This results in standard functions like date, strftime unable to process this datetime string when writing queries.

Wiki should cover:

(1) Brief description of the project (like in the README)
(2) Steps for installation and usage (like in the README)
(3) Data models and schema
(4) Explanation of the test plans
(5) Future plans for the project
etc...

Default main function provided runs only a single plan. We may want to run multiple plans and then compare the plan results. Currently, users need to write their own driver code to run the solver multiple times and generate multiple plans - provide this as standard functionality.

If the "default_plan" option is provided, then run the solver only for that plan; else, generate the plan for every single plan in the plan table. Make sure that planning logic filters by plan (when checking for demand, resource availability etc.). Similarly, export of solution must also be plan specific.

Current logic to write taskplans to the database clears out the table first; but that may not be desirable in workflows where plans are run separately. Need to at the very least define/suggest a process for retaining older plans for later reference/comparison.

Also, currently, demand satisfaction date may be different from requested date; but this information is not being written back into the database. We need a new "demandplan" table to capture this information. That will help us generate a lateness/shortness report easily and also compare multiple plan outputs in the database.

Currently, we are keying the alternate workcenters by the priority value; if the data contains two records with different workcenters, but the same priority value, then the record that is processed second will overwrite the first one!

The result is catastrophic in that it reduces the number of alternate workcenters available for planning and can result in unnecessary tardiness of demand. To see this, consider a modified version of test0016 where we have demand that is planned late. Since we have alternate workcenters, one part is planned on the primary workcenter and the other is planned on the alternate workcenter. We plan both parts late, but on the same dates and minimize the overall tardiness of demand.

However, if the alternate resources are given the same priority, both parts will now see only a single available resource and plan late. For example, test0016 as is has the following output:

Demands:
PART001-DMD001; Priority: 1; Due: 270 on 2018-01-09T00:00; Planned: 270 on 2018-01-10T00:00
PART002-DMD002; Priority: 2; Due: 270 on 2018-01-09T00:00; Planned: 270 on 2018-01-10T00:00

TaskPlans:
PART001-OP10 [ 2018-01-01T00:00 - 2018-01-02T00:00] Qty: 270; DMD001; Loads: RES001
PART001-OP20 [ 2018-01-02T00:00 - 2018-01-03T00:00] Qty: 270; DMD001; Loads: RES003
PART001-SHIP [ 2018-01-03T00:00 - 2018-01-10T00:00] Qty: 270; DMD001; Loads: null
PART002-OP10 [ 2018-01-01T00:00 - 2018-01-02T00:00] Qty: 270; DMD002; Loads: RES002
PART002-OP20 [ 2018-01-02T00:00 - 2018-01-03T00:00] Qty: 270; DMD002; Loads: RES004
PART002-SHIP [ 2018-01-03T00:00 - 2018-01-10T00:00] Qty: 270; DMD002; Loads: null

Note that PART001 loads RES001 and RES003 on the same dates that PART002 loads RES002 and RES004. Demands are satisfied on the same date.

However, if we simply modify the taskworkcenterassn.csv file to have a priority of 1 for all records, the we get the following result:

Demands:
PART001-DMD001; Priority: 1; Due: 270 on 2018-01-09T00:00; Planned: 270 on 2018-01-10T00:00
PART002-DMD002; Priority: 2; Due: 270 on 2018-01-09T00:00; Planned: 270 on 2018-01-11T00:00

TaskPlans:
PART001-OP10 [ 2018-01-01T00:00 - 2018-01-02T00:00] Qty: 270; DMD001; Loads: RES002
PART001-OP20 [ 2018-01-02T00:00 - 2018-01-03T00:00] Qty: 270; DMD001; Loads: RES004
PART001-SHIP [ 2018-01-03T00:00 - 2018-01-10T00:00] Qty: 270; DMD001; Loads: null
PART002-OP10 [ 2018-01-02T00:00 - 2018-01-03T00:00] Qty: 270; DMD002; Loads: RES002
PART002-OP20 [ 2018-01-03T00:00 - 2018-01-04T00:00] Qty: 270; DMD002; Loads: RES004
PART002-SHIP [ 2018-01-04T00:00 - 2018-01-11T00:00] Qty: 270; DMD002; Loads: null

Now note that both OP10's are planned on RES002 and both OP20's are planned on RES004. In effect, RES001 and RES004 have been "deleted" and are not considered as available for planning. As a result, the demand for PART002 is satisfied late.

We need to get both demands planned on 2018-01-10 like in test0016 even if the priority for all resources is the same.

Debug printing code and any other logging code needs to be enclosed within conditionals based on a preset logging/debug level. The level of information printed will depend on this value.

Need to find a nice and intuitive way to do this without any hardcoding of numbers etc.

Refine some of the debugging statements (both details and levels). Commit diff will be self explanatory. This issue is just a placeholder to reference the commit to.

Test out the current code trying to load from mysql and mariadb databases. Make and suitable changes to schema, data files and code to allow compatibility to use any one of sqlite, postgresql, mysql, and mariadb. Capture any changes in the comments of this issue (and in the commit message).

• Make the Plan class default (public-package) access; instead, provide a helper function, getPlanIDs to the JobShop class to get the unique string ID of each plan in the JobShop model; that is sufficient for a user of the package to generate the plan for one of multiple plans in the model.

• Remove all assertions in the code; replace with suitable error messages to either the LOG/badFile and/or stderr/stdout; terminate program where necessary.

• Remove all assertions from code where the input_mode is DATABASE; in these cases, the data should already be relatively clean because of the foreign-key constraints in the database schema. The checks are redundant.

Make the naming conventions match; reformat the directory numbering schemes to allow for >999 tests.

Current schema already allows for modeling lotsizing (both min and max requirements). However, the planning logic does not account for this. We need to enhance the planning logic to take lotsizing into account.

Create a basic algorithm for capacity constrained planning. More details to be added to this bug as the default algorithm is defined and implemented.

Create a sample script and wiki article to demonstrate how the test plan flatfiles can be loaded into a sqlite database.

Create a very simply browser based utility for visualizing the plan. This will require:

(1) An extra output file (in .json format) containing details of the calendar shifts, demands, workcenters, and taskplans

(2) A html file with supporting javascript and css to import the .json plan and display a resource utilization and demand plan view of the plan generated by the jobshop solver.

Basic support for Work in Progress is required for daily use of the software. The previous day's plan will result in releasing work orders to the shop floor for some of the tasks. However, the demand is still not planned/shipped.

The next day, we must take into account these released work orders while planning for demand. It is possible that some of these demands may get canceled or postponed in which case these work orders will still need to be completed but are not reserved for any demand; they are free to be "attached" to any new demand for that part.

This is a placeholder issue. More details on the requirments/design to be added later.

Come up with a suitable format for the output files for test plans for regression testing of functionality. This should include:

(1) Sorting order of output
(2) Metrics that will be compared
(3) How to check for multiple/alternate solutions
(4) Test success/failure reporting