Egeria-IGC-connectivity issues - Part I - Connectivity to IGC can't be restored after IGC restart about egeria-connector-ibm-information-server

[cmgrote]

Thanks for raising this -- I'll have a look ASAP (probably a little later this week). For now, transferring to separate IGC connector repository.

[cmgrote]

@jan-frommann couple of follow-up questions:

1. From the canonical names in the stack traces (eg.org.odpi.openmetadata.adapters.repositoryservices.igc.clientlibrary.IGCRestClient), it appears that you're using the old version of the IGC connector. In the latest version, these should be eg. org.odpi.egeria.connectors.ibm.igc.clientlibrary.IGCRestClient. Any chance you could reproduce with this latest version of the connector to see if the problem has already been addressed? (See instructions at https://github.com/odpi/egeria-connector-ibm-igc -- specifically copying the packaged version of the connector you either build yourself or download from the releases of this repository into a particular location, and referencing this at OMAG Server Platform startup using the -Dloader.path=... parameter.) If you're using the latest version of the vdc helm charts from the core Egeria repository, I think this should already be done for you (ie. should already be using this new connector) (?)
2. Generally a 400: Bad request response back from IGC would indicate that the request being made by the connector includes some parameters that the IGC environment doesn't understand / recognise. This will be the case if, for example, the connector thinks it is using an IGC environment at version x but the IGC environment is actually version y. The auto-detection should handle this, but only occurs at startup of the connector (the OMAG Server Platform admin API's instance call)... I assume when you restart your IGC environment you're not actually switching it to a different version or the like? (If so, this is likely the culprit, and was never really an anticipated scenario to try to work with.) You could force re-detection by restarting the connector itself: you should be able to do this without needing to re-deploy the entire Helm chart, but to do so you'll need to first save the server configuration from the connector's pod. Copy the *.registrystore and omag.* files out of the pod to back them up, delete the pod (to have it re-created), then copy these files into that pod, and then re-run the OMAG Server Platform admin API's instance call against the proxy and it should pickup these configurations again automatically.

[cmgrote]

Trying to reproduce with the following setup:

• Information Server (IGC) running on host A (specifically version 11.5.0.2 SP5)
• Egeria (odpi/egeria:latest image) running under standalone Docker on host B
• Network resolution setup between host A and host B (no firewall in between)

Steps taken:

1. Startup IGC on host A.
2. Startup Egeria (docker container) on host B using:

   docker run -d --name egeria --mount type=bind,source="$LOADER_PATH",target=/opt/egeria/connectors -p 8080:8080 -e LOADER_PATH=/opt/egeria/connectors -e STRICT_SSL=false -e LOGGING_LEVEL_ROOT=INFO odpi/egeria
   

3. Configure Egeria using the sample payloads for the IGC connector setup.
4. Create a simple metadata object in IGC (a new term with name "test123").
5. Observe from logs of Egeria that the term is picked up by the event mapper.
6. Run a search via the Egeria API (eg. /instances/entities/by-property where displayName contains Test) to confirm that the new GlossaryTerm can be found through the connector (1 result).
7. Shutdown IGC on host A. Reboot host A to ensure no stale threads or processes were left behind.
8. Observe from logs of Egeria that the Kafka connection for the event mapper has gone offline:

   org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-1, groupId=IGCOMRSRepositoryEventMapper_consumer] Connection to node 1 (hostname/xxx.xxx.xxx.xxx:xxxx) could not be established. Broker may not be available.
   

9. Startup IGC on host A.
10. Observe from logs of Egeria that the Kafka connection for the event mapper has automatically been restored:

    o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-1, groupId=IGCOMRSRepositoryEventMapper_consumer] Group coordinator hostname:xxxx (id: 2147483646 rack: null) is unavailable or invalid, will attempt rediscovery
    o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-1, groupId=IGCOMRSRepositoryEventMapper_consumer] Discovered group coordinator hostname:xxxx (id: 2147483646 rack: null)
    

11. Create another simple metadata object in IGC (another new term with name "test321").
12. Note from logs of Egeria that the previous session it held is no longer valid, due to the restart of the IGC system, but that it automatically retries with a new session:

    o.o.e.c.i.i.clientlibrary.IGCRestClient  : Request failed -- session may have expired, retrying...
    

13. Observe from logs of Egeria that the new term is picked up by the event mapper.
14. Run a search via the Egeria API (eg. /instances/entities/by-property where displayName contains Test) to confirm that both the newest and older GlossaryTerms can be found through the connector (2 results).

Note that at no point was the Egeria runtime restarted or reconfigured.

As such, I'm unable to reproduce the issue. If still not working, can you provide updated logs (using the latest publicly-available odpi/egeria docker image)?

[cmgrote]

I believe this is now resolved: please re-open with further details (updated logs) if still an issue.