marklogic / ml-app-deployer Goto Github PK
View Code? Open in Web Editor NEWJava client for the MarkLogic REST Management API
License: Other
Java client for the MarkLogic REST Management API
License: Other
https://docs.marklogic.com/REST/management/database-rebalancer
Manage = provide support for creating and updating partitions. Should require the following new classes:
We'll need a directory structure that allows for associating a payload with a database, as the payload itself cannot specify the database. I think this should do the trick:
ml-config/partitions/(name-of-database)/partition1.json
ml-config/partitions/(name-of-database)/partition2.json
The rebalancer endpoints are out of scope for this ticket, just need support for creating and updating partitions.
As an ML developer
I want to TITLE
So that my project can have multiple directories with modules in them, similar to how Roxy has a /src directory and a /rest-api directory
As an ML developer
I want to TITLE
So that I don't have to manually stub out the config and modules directories, and I also want basic implementations of common config files like for a REST API server and for a content database
Ideally, nothing is done as the admin user except for creating some initial roles and users for an app. The rest of a deployment is then done as an app-specific user with the necessary roles and privileges.
I'm writing some unit tests using the QueryManager and predefined search options. Apparently, the namespace uri contains the name of the appserver.
/Default//rest-api/options/default.xml
If I run the QueryManager Java Client API class against the test app server, then it is going to look for the following uri.
/Default/-test/rest-api/options/default.xml
The options are executed against the app server and not the test app server.
DefaultTokenReplacer should allow for a Properties object to be passed in so it's easy for e.g. a Gradle user to support custom tokens and values.
As an ML developer
I want to TITLE
So that I don't have to stub out a triggers database file that will probably only ever have the database name in it
As an ML developer
I want to TITLE
So that I don't have to manage databases manually or create multiple Gradle tasks to get the job done
DeployForestsCommand has a bug where it's trying to create forests with the same names on each host
As a user
I want to TITLE
So that I can share Query Console workspaces with other developers
Routinely, I want to share a bunch of xquery snippets with customers and have that show up automatically in a qc workspace. This would be a really neat feature for starters.
/project
/src
/test
/xqy
/qconsole
/
query1.xqy
query2.js
:
As an ML developer
I want to TITLE
So that I can easily ensure that any modules that have been deleted from version control are no longer in my modules database
http://docs.marklogic.com/REST/PUT/v1/config/properties
This is loaded via the client API, so this could just be part of ml-javaclient-util.
Just using local-name won't work when two amps have the same local-name but reference different modules databases. Will need to do a GET on all those fields to get the system ID for the amp.
As an ML developer
I want to TITLE
So that I get the quicker performance of XCC vs HTTP
Not going to cover pull configurations in this ticket, will do that in a future one. Only going to support configs and targets.
As a ML developer
I want to TITLE
So that I don't have to create users manually or define a task for each one
Needed for a project where a dependency needs to contribute its own config. I think AppConfig will need a List, and commands will need to figure out what to do with that list.
In general, we'll want to load the dependency config first. The content database is different - we'll still do a merge there.
As a ML developer
I want to TITLE
So that I don't have to create it manually or define a Gradle task for each one
As a ML developer
I want to TITLE
So that I don't have to create roles manually or define a task for each one
As an ML developer
I want to TITLE
So that I have a reliable way for completely removing the app server, modules database, and content database created by /v1/rest-apis
AdminManager currently has a very simple impl that just calls /admin/v1/timestamp until it doesn't catch an error. A more robust impl would care about the last restart timestamp being updated and handle exceptions more gracefully.
Some code I had written before on this, but it wasn't quite working - it needs to at least catch a ResourceAccessException that is thrown by RestTemplate when ML isn't available - i.e. when no HTTP response is returned:
public void waitForRestart(String lastRestartTimestamp) {
logger.info("Ignore any HTTP client logging about socket exceptions and retries, those are expected while waiting for MarkLogic to restart");
try {
while (true) {
sleepUntilNextRestartCheck();
try {
String restart = getLastRestartTimestamp();
if (restart != null && !restart.equals(lastRestartTimestamp)) {
logger.info(String
.format("MarkLogic has successfully restarted; new restart timestamp [%s] is greater than last restart timestamp [%s]",
restart, lastRestartTimestamp));
break;
}
} catch (HttpServerErrorException hsee) {
if (HttpStatus.SERVICE_UNAVAILABLE.equals(hsee.getStatusCode())) {
logger.warn("Waiting for restart, service unavailable, will keep waiting");
} else {
throw hsee;
}
}
}
} catch (Exception e) {
String message = "Caught exception while waiting for MarkLogic to restart: " + e.getMessage();
if (logger.isDebugEnabled()) {
logger.warn(message, e);
} else {
logger.warn(message);
}
}
}
Need a way to create a certificate template and then refer to it by ID in the JSON/XML for a server.
That means whenever a resource is created, we want to save in CommandContext - possibly in AppConfig's map of tokens - the name of the created resource and the ID that was generated for it. That won't apply for all resources, but we should do it for the ones it applies for.
For a template named "sample-app-template", the server file could then have the following in it:
"ssl-certificate-template": "%%certificate-template-id:sample-app-template%%"
Need this ability to allow for the /v1/rest-apis call to use an existing database that has forests configured in a specific manner.
For modules/triggers/schemas database, we generally only need one primary forest in a cluster
As a ML developer
I want to TITLE
So that I don't have to do this manually or write a Gradle task to get it done
As an ML developer
I want to TITLE
So that my build script can delete multiple servers successfully
The tests for ml-app-deployer currently assume a username/password combo of admin/admin. This won't always be the case, and so we should first read in config from gradle.properties using ml-junit, and then we should allow for an optional, non-version-controlled file to be read in to override what's in gradle.properties.
Needed for an undeploy
As an ML developer
I want to TITLE
So that I don't have to manually configure bitemporal features in an application
This is needed so that an app server that references the certificate template will actually function as an SSL server.
This allows for the Admin app to work on each node.
As an ML developer
I want to TITLE
So that I can split my database configuration across multiple JSON files, as I don't always want to include all of them based on what I'm doing
As an ML developer
I want to TITLE
So that I don't have to create it manually nor create a Gradle task for each one
Can wait till 2.1 for this, groups aren't commonly used.
As a ML developer
I want to TITLE
So that I don't have to set it up manually or define Gradle tasks for each domain and pipeline
As a user
I want to TITLE
So that I can allow users to execute assets without having to have the rest-admin role
http://docs.marklogic.com/REST/PUT/v1/ext/[directories]/[asset]
2nd parameter - perm:{role}*
As an ML developer
I want to TITLE
So that I don't have to do this manually, and so I can also ensure this happens at the right time between all the other deployment steps
As a ML developer
I want to TITLE
So that I don't have to do this manually nor create Gradle tasks to get it done
As an ML developer
I want to TITLE
So that I don't have to manually setup a triggers database for my project
As an ML developer
I want to TITLE
So that I can get this done via the command line instead of having to click through the admin tool
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.