lsflk / gig

General Information Graph

Agency Jekyll theme

Agency theme based on Agency bootstrap theme

How to use

###Portfolio

Portfolio projects are in '/_posts'

Images are in '/img/portfolio'

###About

Images are in '/img/about/'

###Team

Team members and info are in '_config.yml'

Images are in '/img/team/'

Demo

View this jekyll theme in action here

========= For more details, read documentation

gig's Issues

When child entities get moved from one parent to another, manage how the change is reflected under 'parent' attribute

when a child is removed, moved between parents the change should be reflected in the child entity

Add pagination support for search api

Allow clients to request paginated results form the search API. Currently it is limiting the results returned to avoid overflow.

disallow changing value if the same value already exist for the exact start_date

for a given attribute of an entity, when a new value is assigned validate the value before assigning to make sure the same value has not been assigned before. Check by RawValue and StartDate, Since the same value can be recurring at different times.

maintain a list of categories separately

Whenever categories are added to an entity. Maintain a separate list of categories, so it can be used as a list categories available.

• when a new entity is created, modified, add the categories to the list if the category is not listed already.

implement API authentication using JWT

Create API to allow terminating an entity

Parameters

1. termination date - effective date of termination. only the entities that have been created before this date will be terminated
2. title, category or any query parameters to narrow down the entities to be terminated.

the function already available in orgchart importer. move the code to GIG

Normalizer as a seperate service

move the normalizer service as a separate service from GIG

multiple entities created for orgchart on rerunning same script-

if an old importer script is executed multiple times a newer entity might get terminated by a termination command on old script, hence creating multiple entities having same name at a given time

Index entity default images for quick retrieval

develop an API to get the default image of an entity using entity title as a parameter. This feature enhances the loading performance.

Update mongodb driver

GIG architecture is built with the capability to extend the data storage/database to any technology. For eg. MongoDB, MySQL, Minio, and Redis. The strategy pattern enables connecting to different database/storage drivers through the same interface. The MongoDB driver currently used is not supported in the official MongoDB atlas. Therefore need to extend the support to the new MongoDB driver by writing a new MongoDB service and new repository classes without overwriting the existing classes. This has been initiated in the https://github.com/LSFLK/GIG/tree/update-mongo-driver branch.

avoid adding same value to attributes

if entity name changes to a existing name. How to merge the two entities

If entity named A changes its name to B, and an entity by the name B already exists. Implement the mechanism to merge the two entities

normalize entity links before saving the entity

normalize entity's links and attach date to the link, so even if the linked entity name changes relation to the liked entity can be established by searching for the enity by title and date of the titles. Feature mention in issue #32

Make a log in entity for user modification

track manual modifications to an entity by a user. Keep a version history for user modifications

write unit tests for the following

• storage->minio get and upload
• file get upload api
• normalizers api

resume crawlers from where stopped

implement features to allow crawlers to resume crawling from last stopped point

Change entity default image selection criteria

Currently when crawling html content, the default image is selected based only on the highest width criteria. modify this function to select default image by highest area.

Get attribute value from entity by date

Get the value of an attribute of the entity by a given date, neglecting any changes made to the attribute after the date specified

api to change an entity name

change entity name from old one to new one.

Limit attributes in search results and by version date

allow client to send list of attribute in search query to include in search results and filtered by a version date. So only attributes values before the given date will be shown. (Can retrieve old version of an entity by date )

Cache frequently requested query results

Cache query results of repetitive queries to lessen the load on the database for data and implement a cache expiration feature based on frequency of change of cached data

Generate snippets for entities on create, update

When a entity is created or the content of an existing entity is modified, generate a content snippet to be shown in the search results. A cron job for tracking updated entities and generating new snippets based on latest content is recommended

If a entity is marked terminated, and if new information emerges for that entity???

when a entity is terminated and a later on time new entity with same name is created. If new information needs to be added the entity. Ambiguity occurs in determining whether the new information should be added to the terminated entity or latest new entity.

Solution: focus on the date of new information and determine the active entity with the entity name and time. Such that the entity which has the name mentioned in new information during the time mentioned in the new information is considered as the entity where the new information should be added.

Remove eager loading entity.

Avoid eager loading related entities from links when loading an entity. If a title of an existing entity is changed, search for all entities linked to the changed entity and change the link to address to new changed entity title.

most recent entities should be limited only to the specific lifetime of the entity name

when searching for an entity by a past name and date, outputs entities that have different names in the given date. Fix this only to output entities within valid date range given

make all class attributes private. make accessible only through getters and setters

Remove minio file caching to a separate storage - eg. Redis, CDN

Currently, file caching is implemented within the API server. This will cause out of storage issues and cause server instances to crash. Move the cache to a separate caching service/ CDN. This would also help to improve resource loading time

Use zap for logging

The current inbuild logging library "log" is slower. Use "go.uber.org/zap" or any other faster and queryable logging mechanism

Cache Normalizer

Cache normalized names to minimize use of Google custom search API and to improve efficiency.

Update missing documentations

• linked api
• related entities api
• entity api

Post crawling feature

When a user search for a entity name and if the entity is not found in the GIG servers, queue up the entity for crawling, so a crawler can later crawl the entity from the web or any other sources and make a new record in the GIG servers

Merge entity content

When information is crawled and retrieved from several sources, implement an mechanism to add the changes and verify overriding existing data. Include source, timestamp, effective date, etc.

define enums for value types