Comments (4)
Thanks for raising this -- I'll have a look ASAP (probably a little later this week). For now, transferring to separate IGC connector repository.
from egeria-connector-ibm-information-server.
@jan-frommann couple of follow-up questions:
- From the canonical names in the stack traces (eg.
org.odpi.openmetadata.adapters.repositoryservices.igc.clientlibrary.IGCRestClient
), it appears that you're using the old version of the IGC connector. In the latest version, these should be eg.org.odpi.egeria.connectors.ibm.igc.clientlibrary.IGCRestClient
. Any chance you could reproduce with this latest version of the connector to see if the problem has already been addressed? (See instructions at https://github.com/odpi/egeria-connector-ibm-igc -- specifically copying the packaged version of the connector you either build yourself or download from the releases of this repository into a particular location, and referencing this at OMAG Server Platform startup using the-Dloader.path=...
parameter.) If you're using the latest version of thevdc
helm charts from the core Egeria repository, I think this should already be done for you (ie. should already be using this new connector) (?) - Generally a
400: Bad request
response back from IGC would indicate that the request being made by the connector includes some parameters that the IGC environment doesn't understand / recognise. This will be the case if, for example, the connector thinks it is using an IGC environment at version x but the IGC environment is actually version y. The auto-detection should handle this, but only occurs at startup of the connector (the OMAG Server Platform admin API'sinstance
call)... I assume when you restart your IGC environment you're not actually switching it to a different version or the like? (If so, this is likely the culprit, and was never really an anticipated scenario to try to work with.) You could force re-detection by restarting the connector itself: you should be able to do this without needing to re-deploy the entire Helm chart, but to do so you'll need to first save the server configuration from the connector's pod. Copy the*.registrystore
andomag.*
files out of the pod to back them up, delete the pod (to have it re-created), then copy these files into that pod, and then re-run the OMAG Server Platform admin API'sinstance
call against the proxy and it should pickup these configurations again automatically.
from egeria-connector-ibm-information-server.
Trying to reproduce with the following setup:
- Information Server (IGC) running on host A (specifically version 11.5.0.2 SP5)
- Egeria (
odpi/egeria:latest
image) running under standalone Docker on host B - Network resolution setup between host A and host B (no firewall in between)
Steps taken:
- Startup IGC on host A.
- Startup Egeria (docker container) on host B using:
docker run -d --name egeria --mount type=bind,source="$LOADER_PATH",target=/opt/egeria/connectors -p 8080:8080 -e LOADER_PATH=/opt/egeria/connectors -e STRICT_SSL=false -e LOGGING_LEVEL_ROOT=INFO odpi/egeria
- Configure Egeria using the sample payloads for the IGC connector setup.
- Create a simple metadata object in IGC (a new
term
with name "test123"). - Observe from logs of Egeria that the term is picked up by the event mapper.
- Run a search via the Egeria API (eg.
/instances/entities/by-property
wheredisplayName
containsTest
) to confirm that the newGlossaryTerm
can be found through the connector (1 result). - Shutdown IGC on host A. Reboot host A to ensure no stale threads or processes were left behind.
- Observe from logs of Egeria that the Kafka connection for the event mapper has gone offline:
org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-1, groupId=IGCOMRSRepositoryEventMapper_consumer] Connection to node 1 (hostname/xxx.xxx.xxx.xxx:xxxx) could not be established. Broker may not be available.
- Startup IGC on host A.
- Observe from logs of Egeria that the Kafka connection for the event mapper has automatically been restored:
o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-1, groupId=IGCOMRSRepositoryEventMapper_consumer] Group coordinator hostname:xxxx (id: 2147483646 rack: null) is unavailable or invalid, will attempt rediscovery o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-1, groupId=IGCOMRSRepositoryEventMapper_consumer] Discovered group coordinator hostname:xxxx (id: 2147483646 rack: null)
- Create another simple metadata object in IGC (another new
term
with name "test321"). - Note from logs of Egeria that the previous session it held is no longer valid, due to the restart of the IGC system, but that it automatically retries with a new session:
o.o.e.c.i.i.clientlibrary.IGCRestClient : Request failed -- session may have expired, retrying...
- Observe from logs of Egeria that the new term is picked up by the event mapper.
- Run a search via the Egeria API (eg.
/instances/entities/by-property
wheredisplayName
containsTest
) to confirm that both the newest and olderGlossaryTerm
s can be found through the connector (2 results).
Note that at no point was the Egeria runtime restarted or reconfigured.
As such, I'm unable to reproduce the issue. If still not working, can you provide updated logs (using the latest publicly-available odpi/egeria
docker image)?
from egeria-connector-ibm-information-server.
I believe this is now resolved: please re-open with further details (updated logs) if still an issue.
from egeria-connector-ibm-information-server.
Related Issues (20)
- Lack of context for data_files without data_file_folders HOT 8
- NPE when trying to get EntityDetail for rid that is not present in IGC
- Implement a NOOP event mapper HOT 1
- findEntities with multiple conditions does not return entities HOT 1
- Add differentiation between primary and secondary categories HOT 1
- Property 'assigned_to_terms' no longer supported for searching using 'main_object' generic type HOT 1
- Implement new classification for primary category HOT 1
- IGC repository connector - validation for instance properties mapped with wrong primitive type HOT 3
- Data Stage connector support for high level lienage HOT 7
- Results for a relationship search by type (with multiple mappings) extend beyond pageSize
- Improve handling of complex search criteria for findEntities method HOT 2
- Searching for NEQ minCardinality on SchemaAttributes does not work
- repository-services getEntityDetail is ignoring MetadataCollectionId part of GUID HOT 2
- Move data_file_field mappings from TabularColumn to TabularFileColumn
- Data Stage connector batch processing sync state per instance
- Possible inconsistent behaviour HOT 2
- Data Stage connector capability to process virtual assets
- Mapping new typedef properties (Failed test ConnectorTest.verifySupportedTypes) HOT 2
- Can igc-adapter create anchor classifications for data asset types?
- SonarCloud Quality Gate Code duplication rule too restrictive HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from egeria-connector-ibm-information-server.