delphix / dx-workshops

Various Delphix Dynamic Data Platform workshops and associated automation artifacts like Packer templates, Terraform blueprints, Ansible Roles, Golang, etc

Various Delphix Dynamic Data Platform Workshops and associated automation artifacts like Packer templates, Terraform blueprints, Golang, etc

• Toolchain Workshops
• Description
• Go Examples
• Base Templates
  • Usage
    • Configuring
    • User Variables
    • Building
• Links
• Contribute
  • Workflow
• Reporting Issues
• Statement of Support
• License

Toolchain Workshops

If you are interested in deploying the Toolchain Workshops, please see the instructions at one of the following links below:

Oracle Toolchain Workshop

Postgres Toolchain Workshop

Description

This repo consists of some standard OS + Database templates configured ready to use with Delphix (located in the packer-templates folder):

• delphix-centos6.9-oracle11.2.0.4.json
• delphix-centos7-ansible-base.json
• delphix-centos7-oracle-11.2.0.4.json
• delphix-centos7-oracle-12.2.0.1.json
• delphix-centos7-postgres-9.6.json
• delphix-ubuntu-bionic-guacamole.json

There are also some bespoke Packer templates used for specific scenarios:

• delphix-centos7-daf-app.json
• delphix-centos7-kitchen_sink.json
• delphix-centos7-tooling-base.json
• delphix-tcw-jumpbox.json
• delphix-tcw-oracle12-source.json
• delphix-tcw-oracle12-target.json
• delphix-tcw-tooling-oracle.json
• delphix-toolchain-dafdb-pgsql-source.json

The Oracle templates requires access to the Oracle binaries placed in an AWS s3 bucket.

Go Examples

The go directory contains examples of interacting with the Delphix Dynamic Data Platform using golang. The examples in that directory are purpose-built to be used in the workshops contained within this repo.

Base Templates

1. Clone this repository
2. Navigate into the cloned directory in a terminal
3. Copy the .example.env to .environment.env

These templates depend on Packer and Ansible existing in your path. If you are running a Mac, then the easiest way is to install via homebrew. After cloning this repo, install the required Ansible dependencies.

brew install ansible packer git
git clone https://github.com/delphix/dx-workshops
cp .example.env .example.env
ansible-galaxy install -r roles/X_requirements.yml

Usage

Configuring

1. Edit the .environment.env file in the root directory of the cloned repo

User Variables

1. AWS_ACCESS_KEY_ID - The AWS_ACCESS_KEY_ID environment variable
2. AWS_SECRET_ACCESS_KEY - The AWS_SECRET_ACCESS_KEY environment variable
3. AWS_REGION - The region packer will build the temporary infrastructure for the AMI
4. ORACLE_BINARIES_ROOT_URL - The URL where the AWS instance can retrieve the Oracle binaries during the packer build (For Oracle templates, only)
5. AWS_VPC_ID - The VPC ID from the region that packer will use
6. AWS_SUBNET_ID - The subnet ID from the VPC that packer will use

The below values are arbitrary, and only for tagging resources

1. AWS_EXPIRATION - The date this AMI is expired, i.e. "2037-07-01" or "never"
2. AWS_OWNER - The name of the person who owns the AMI, i.e. "Adam Bowen"
3. AWS_PROJECT - The name of the project that the AMI belongs, or came, from
4. AWS_COSTCENTER - The name of the cost center, if applicable

Building

1. source the .environment.env file
2. run packer against the template you want to use:

source .example.env
cd packer-templates
packer build delphix-centOS6.9-oracle11.2.0.4.json

packer build delphix-centOS6.9-oracle11.2.0.4.json 
cent69-Oracle11204 output will be in this color.

==> cent69-Oracle11204: Force Deregister flag found, skipping prevalidating AMI Name
    cent69-Oracle11204: Found Image ID: ami-8b44f234
==> cent69-Oracle11204: Creating temporary keypair: packer_5ad55a87-66df-e148-9439-a7bd06aa04fb
==> cent69-Oracle11204: Creating temporary security group for this instance: packer_5ad55ad1-10d5-7eda-e077-9741925ce7e4
==> cent69-Oracle11204: Authorizing access to port 22 from 0.0.0.0/0 in the temporary security group...
==> cent69-Oracle11204: Launching a source AWS instance...
==> cent69-Oracle11204: Adding tags to source instance
    cent69-Oracle11204: Adding tag: "Name": "Packer Builder"
    cent69-Oracle11204: Instance ID: i-0434f9b2e3a90c0c3
==> cent69-Oracle11204: Waiting for instance (i-0434f9b2e3a90c0c3) to become ready...
==> cent69-Oracle11204: Waiting for SSH to become available...

Second Example:

source .dafdb-source
cd packer-templates
packer build delphix-toolchain-dafdb-source.json

packer build delphix-toolchain-dafdb-source.json
delphix-toolchain-dafdb-source output will be in this color.

==> delphix-toolchain-dafdb-source: Force Deregister flag found, skipping prevalidating AMI Name
    delphix-toolchain-dafdb-source: Found Image ID: ami-090c2433423df7c1b
==> delphix-toolchain-dafdb-source: Creating temporary keypair: packer_5c24fd34-75a9-e10e-afa8-5784ac498ae2
==> delphix-toolchain-dafdb-source: Creating temporary security group for this instance: packer_5c24fd36-f5cc-8180-51b5-1bc18409d591
==> delphix-toolchain-dafdb-source: Authorizing access to port 22 from 0.0.0.0/0 in the temporary security group...
==> delphix-toolchain-dafdb-source: Launching a source AWS instance...
==> delphix-toolchain-dafdb-source: Adding tags to source instance
    delphix-toolchain-dafdb-source: Adding tag: "dlpx:CostCenter": "305000 - Development Engineering"

Links

Include useful links to references or more advanced guides.

• Packer Intro
• Building AMIs with Packer

Contribute

All contributors are required to sign the Delphix Contributor Agreement prior to contributing code to an open source repository. This process is handled automatically by cla-assistant. Simply open a pull request and a bot will automatically check to see if you have signed the latest agreement. If not, you will be prompted to do so as part of the pull request process.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

Workflow

1. Fork the project.
2. Make your bug fix or new feature.
3. Add tests for your code.
4. Send a pull request.

Contributions must be signed as User Name <[email protected]>. Make sure to set up Git with user name and email address. Bug fixes should branch from the current stable branch. New feature should be based on the master branch.

Reporting Issues

Issues should be reported in the repo's issue tab.

Statement of Support

This software is provided as-is, without warranty of any kind or commercial support through Delphix. See the associated license for additional details. Questions, issues, feature requests, and contributions should be directed to the community as outlined in the Delphix Community Guidelines.

License

This is code is licensed under the Apache License 2.0. Full license is available here.

dx-workshops's Issues

Python installed in workshop AMI's should be upgraded to 3.x

Problem

Python 2.7 is EOL on Jan 1, 2020

Solution

Upgrade and test Python 3.x

Alternatives

Continue to use 2.7, but will have to use backports/deprecated repositories. Systems will be vulnerable and fragile.

Additional Info

https://pythonclock.org/

masking chrome bookmark should be update to masking login page

Problem

The chrome bookmark for masking currently points to the virtualization admin interface and not /masking

Solution

append /masking to the bookmark

Alternatives

Do nothing and let people figure it out on their own.

Additional Info

Include any additional information that might be relevant.

Tag AMI's with Release for easier filtering

Problem

As this repo evolves, users may want to easily choose which version of the workshops to deploy/cleanup, etc.

Solution

Apply any applicable release tags to AMI's created by this repo, and add applicable filtering options to the container functions (build, deploy, cleanup, etc)

Alternatives

Do nothing and manually filter/deploy via AWS, which is untenable.

Additional Info

Include any additional information that might be relevant.

AWS secrets piped into log

Debugging dev code is still present that posts AWS secrets into the log

Race Condition on jumpbox boot

Expected Behavior

The provision.sh should execute after the startup.sh

Actual Behavior

In some cases, the startup.sh takes longer to complete and exposes a race condition (especially when how_many>1)

Steps to Reproduce the Problem

Race condition is always there. The problem manifests itself by the provision script beginning to connect to a Delphix engine before the hosts file is updated.

Version

all

Masking configuration and jobs in classrooms are hardcoded to 10.0.1.X

Expected Behavior

Masking configuration and jobs should be configured to represent the network configuration in which they are deployed. This will cause masking jobs to not behave as expected, or possibly fail.

Actual Behavior

Masking engine is hardcoded as 10.0.1.11 and jdbc connection string is hardcoded to 10.0.1.20

Steps to Reproduce the Problem

Deploy as-is

Version

=<v2.12

Idea: Jumpbox splash screen should increment version with release

The jumpbox splash screen shows version 2.0. Would be great if the splash screen showed precise version of the build (i.e. 2.09)

Cannot find module './child/index'

On workshop boot, the jumpbox runs a script called provision.sh. This script builds and deploys web applications to the web servers.

Occasionally, the ansible role that builds and deploys the web app will fail on the "BUILD HTML" step due to a failure in "INSTALL PACKAGES" to properly pull down packages.

The result is that the WAIT file will turn to ERROR with error messaging like the below in the log:

fatal: [testweb]: FAILED! => {"attempts": 5, "changed": true, "cmd": ["/usr/local/lib/npm/bin/ng", "build", "--prod"], "delta": "0:00:02.927936", "end": "2019-09-18 05:34:46.951764", "msg": "non-zero return code", "rc": 1, "start": "2019-09-18 05:34:44.023828", "stderr": "Your global Angular CLI version (8.2.1) is greater than your local\nversion (6.1.5). The local Angular CLI version is used.\n\nTo disable this warning use "ng config -g cli.warnings.versionMismatch false".\nCannot find module './child/index'\nError: Cannot find module './child/index'\n at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)\n at Function.resolve (internal/modules/cjs/helpers.js:33:19)\n at Object. (/tmp/test/client/node_modules/worker-farm/lib/fork.js:4:30)\n at Module._compile (internal/modules/cjs/loader.js:778:30)\n at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)\n at Module.load (internal/modules/cjs/loader.js:653:32)\n at tryModuleLoad (internal/modules/cjs/loader.js:593:12)\n at Function.Module._load (internal/modules/cjs/loader.js:585:3)\n at Module.require (internal/modules/cjs/loader.js:692:17)\n at require (internal/modules/cjs/helpers.js:25:18)\n at Object. (/tmp/test/client/node_modules/worker-farm/lib/farm.js:15:33)\n at Module._compile (internal/modules/cjs/loader.js:778:30)\n at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)\n at Module.load (internal/modules/cjs/loader.js:653:32)\n at tryModuleLoad (internal/modules/cjs/loader.js:593:12)\n at Function.Module._load (internal/modules/cjs/loader.js:585:3)", "stderr_lines": ["Your global Angular CLI version (8.2.1) is greater than your local", "version (6.1.5). The local Angular CLI version is used.", "", "To disable this warning use "ng config -g cli.warnings.versionMismatch false".", "Cannot find module './child/index'", "Error: Cannot find module './child/index'", " at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)", " at Function.resolve (internal/modules/cjs/helpers.js:33:19)", " at Object. (/tmp/test/client/node_modules/worker-farm/lib/fork.js:4:30)", " at Module._compile (internal/modules/cjs/loader.js:778:30)", " at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)", " at Module.load (internal/modules/cjs/loader.js:653:32)", " at tryModuleLoad (internal/modules/cjs/loader.js:593:12)", " at Function.Module._load (internal/modules/cjs/loader.js:585:3)", " at Module.require (internal/modules/cjs/loader.js:692:17)", " at require (internal/modules/cjs/helpers.js:25:18)", " at Object. (/tmp/test/client/node_modules/worker-farm/lib/farm.js:15:33)", " at Module._compile (internal/modules/cjs/loader.js:778:30)", " at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)", " at Module.load (internal/modules/cjs/loader.js:653:32)", " at tryModuleLoad (internal/modules/cjs/loader.js:593:12)", " at Function.Module._load (internal/modules/cjs/loader.js:585:3)"], "stdout": "", "stdout_lines": []}

PLAY RECAP *********************************************************************
testweb : ok=15 changed=4 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0

3117
It took 407 seconds to complete

FULL Dataset bookmark not present in Dev/QA Catalog

Ubuntu-based images fail in the build process with "Unable to acquire the dpkg frontend lock"

Expected Behavior

The build process should successfully stop the unattended upgrade service, keeping it from locking up the dpkg.lock

Actual Behavior

The apt-daily.service is also running in the background and doing the same

Steps to Reproduce the Problem

Rebuild the jumpbox ami from an old guacamole base

Version

=>2.11

Additional Information

[0;32m    delphix-tcw-jumpbox: FAILED - RETRYING: Install Desktop tools (1 retries left).�[0m
�[0;32m    delphix-tcw-jumpbox: fatal: [default]: FAILED! => {"attempts": 5, "cache_update_time": 1571912950, "cache_updated": false, "changed": false, "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\"      install 'putty' 'remmina' 'postgresql-client-common' 'postgresql-client' 'caffeine' 'imagemagick'' failed: E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n", "rc": 100, "stderr": "E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n", "stderr_lines": ["E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)", "E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?"], "stdout": "", "stdout_lines": []}�[0m

ubuntu@ip-172-31-51-141:~$ ps -ef | grep apt
root      2164     1  0 11:32 ?        00:00:00 /bin/sh /usr/lib/apt/apt.systemd.daily install
root      2180  2164  0 11:32 ?        00:00:00 /bin/sh /usr/lib/apt/apt.systemd.daily lock_is_held install
ubuntu    2517  2479  0 11:32 ?        00:00:00 /bin/sh -c /bin/sh -c 'sudo -H -S -n  -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-vtpgtwpzodplqmognbszsqqglfqufvgu ; /usr/bin/python /home/ubuntu/.ansible/tmp/ansible-tmp-1571916755.57-71146696876328/AnsiballZ_apt.py'"'"' && sleep 0'
ubuntu    2518  2517  0 11:32 ?        00:00:00 /bin/sh -c sudo -H -S -n  -u root /bin/sh -c 'echo BECOME-SUCCESS-vtpgtwpzodplqmognbszsqqglfqufvgu ; /usr/bin/python /home/ubuntu/.ansible/tmp/ansible-tmp-1571916755.57-71146696876328/AnsiballZ_apt.py' && sleep 0
root      2519  2518  0 11:32 ?        00:00:00 sudo -H -S -n -u root /bin/sh -c echo BECOME-SUCCESS-vtpgtwpzodplqmognbszsqqglfqufvgu ; /usr/bin/python /home/ubuntu/.ansible/tmp/ansible-tmp-1571916755.57-71146696876328/AnsiballZ_apt.py

provisioning script is completing successfully, though delphix job errors out

The events that transpired in the log snippet below should have resulted in the ERROR file being created, and not READY

RESTY 2019/10/02 07:07:54 ERROR Post https://delphixengine/resources/json/delphix/session: dial tcp 10.0.1.10:443: connect: connection refused, Attempt 1
RESTY 2019/10/02 07:07:59 ERROR Post https://delphixengine/resources/json/delphix/session: dial tcp 10.0.1.10:443: connect: connection refused, Attempt 2
time="2019-10-02T07:08:10Z" level=info msg="Successfully Logged in"
time="2019-10-02T07:08:10Z" level=info msg="Searching for Environment by name"
time="2019-10-02T07:08:10Z" level=info msg="Found proddb: UNIX_HOST_ENVIRONMENT-3"
time="2019-10-02T07:08:10Z" level=info msg="Searching for Host by Environment"
time="2019-10-02T07:08:10Z" level=info msg="Found proddb: UNIX_HOST_ENVIRONMENT-3"
time="2019-10-02T07:08:10Z" level=info msg=10.0.1.20
time="2019-10-02T07:08:11Z" level=info msg="Searching for Environment by name"
time="2019-10-02T07:08:11Z" level=info msg="Found devdb: UNIX_HOST_ENVIRONMENT-4"
time="2019-10-02T07:08:11Z" level=info msg="Searching for Host by Environment"
time="2019-10-02T07:08:11Z" level=info msg="Found devdb: UNIX_HOST_ENVIRONMENT-4"
time="2019-10-02T07:08:11Z" level=info msg=10.0.1.30
time="2019-10-02T07:08:11Z" level=fatal msg="{\"type\":\"ErrorResult\",\"status\":\"ERROR\",\"error\":{\"type\":\"APIError\",\"details\":\"The operation could not be completed because one or more objects are replicated. Replicated objects cannot be modified.\",\"action\":\"Failover the namespace and try again.\",\"id\":\"exception.executor.replica.update\",\"commandOutput\":null,\"diagnoses\":[]}}"
Pseudo-terminal will not be allocated because stdin is not a terminal.
Cloning into '/tmp/dev'...
done.
Cloning into '/tmp/test'...
done.
Cloning into '/tmp/prod'...
done.
9645

Jenkins jobs intermittently failing with durable_task_monitor errors

Expected Behavior

Jobs should complete normally

Actual Behavior

Intermittently, jobs are failing with java and golang based durable_task_monitor errors.

Steps to Reproduce the Problem

Run enough jobs, and it will happen. Seems more prevalent under system load.

Version

=< 2.13

Additional Information

This issue only recently surfaces. The timing coincides with a recent update to the durable_task_monitor plugin on October 22nd. https://github.com/jenkinsci/durable-task-plugin/blob/master/CHANGELOG.md The change included some major changes.

`validate` should cite the specific version of the Delphix Engine that was found.

Problem

docker-compose run tcw validate validates a Delphix Engine matching the glob is found, but does not specifically state which version of Delphix is found

Solution

Return the version of Delphix that is found and would be used in the deploy

Alternatives

Do nothing and just look in the AWS console to determine which Delphix Engine version would be used.

Additional Info

WARNING: The CURRENT_UID variable is not set. Defaulting to a blank string.
Validating environment variables

Checking that the Delphix Engine 5.3.5.* is shared with this account in the us-west-2 region...
Found Delphix Engine 5.3.5.*

Upgrade docker container to use Python 3

Problem

Python 2.7 is EOL on Jan 1, 2020

Solution

Upgrade and test Python 3.x

Alternatives

Continue to use 2.7, but will have to use backports/deprecated repositories. Systems will be vulnerable and fragile.

Additional Info

https://pythonclock.org/

Include option to disable retention policies

Problem

If you keep TCW's around for a couple of weeks, or more, then the default retention policies will clean up the logs. This is normal behavior. But, the unintended consequence is that it can leave the workshop participant in a state where they cannot continue and receive errors (because they do not know to create a new snapshot)

Solution

Add an option to disable retention policy in the virtualization engine. By default, it will be set to true.

Alternatives

Write documentation advising of this possible condition and rely on the participant to "repair" the workshop state.

Additional Information

User may see errors in the Delphix engine logs like the following:

command: umask 027; . $DB_SCRIPT_DIR/setup-oraenv.sh; $DLPX_SHELL $DB_SCRIPT_DIR/doOpenPDB.sh /mnt/provision/DEVDB-tcdb-0zwqSE3qciZJTWb6/datafile
PATMM 536870912 restricted output:END_OF_SETUP Virtual Pluggable Database PATMM cannot be opened to READ WRITE or READ WRITE RESTRICTED mode - status = ‘MOUNTED’ Session altered. Session altered. Session altered. alter pluggable database PATMM open read write
restricted * ERROR at line 1: ORA-65106: Pluggable database #5 (PATMM) is in an invalid state. OPEN_PDB_ERROR

Postgres Source Config needs to be updated on class preparation

Expected Behavior

The Postgres dsource requires a staging server configuration and this should be updated as a part of the tw_prep process

Actual Behavior

The source config remains in its original configuration

Steps to Reproduce the Problem

Provision a workshop

Version

=< 3.0RC3

Create a workshop that is not based on Oracle

Problem

The current TCW is based on Oracle 12. This requires customers to furnish their own Oracle binaries and supply a license.

Solution

Create a variant based on postgres

Alternatives

Do nothing or use another free db like mysql

Library methods not exiting immediately on error in Jenkins pipeline

Expected Behavior

Errors in the library methods should cause the pipeline to enter a failed state

Actual Behavior

Methods are continuing to execute as if errors did not occur.

Steps to Reproduce the Problem

Rename a packer.json file and build. The logs show an error, the build continues

Version

=> v2.13

Additional Information

The issue is the set builtin in the methods is currently +x which requires the specification of -e

Pin Jenkins Plugin versions in workshop build

Problem

As of now, plugins are always installed from "latest" on workshop build. This can lead to unstable changes, as seen in #23

Solution

Modify kitchen_sink.yaml to include plugin versions in list.

Alternatives

Do nothing, and roll the dice.

Additional Info

Include any additional information that might be relevant.

Error waiting for instance

Expected Behavior

Packer starts instances, waits for them to become ready, and does.

Actual Behavior

In some cases, packer gives up waiting on the instance to be ready.

Steps to Reproduce the Problem

Build a workshop, psuedo-random error

Version

=<3.0.0

Additional Information

Error waiting for instance (i-01d83a2ec67c70956) to become ready: ResourceNotReady: failed waiting for successful resource state�

Need to RCW further when this is happening to discover if a timeout setting needs to be adjusted or if there is possibly an issue with AWS.

Validate currently does not consider S3_ env params

Expected Behavior

Validate should use the S3_* env parameters to validate the existence of s3 objects, when specified

Actual Behavior

Validate uses the AWS_* credentials and results in ```An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied (validate function)

Steps to Reproduce the Problem

specify an s3 bucket that the access/secret key specified in the AWS* credentials cannot access, but the S3_ env vars can.

Version

All

Pin ansible role versions

Problem

Ansible roles are currently not pinned for versions. This could potentially lead to a breaking change.

Solution

pin the role versions in the requirements yaml

Alternatives

Do nothin. Roll the dice.

Additional Info

Include any additional information that might be relevant.

Move external dependencies list to workshop.yaml file

Problem

Right now, the external files that are required to support the building of the workshops are stored in a shell variable S3_BINARIES in the validate.sh. This makes the file fragile and does not lend itself for reuse. It also requires rebuilding the docker container anytime a workshop dependency is updated.

Solution

Move the external dependencies into the workshop.yaml file as this describes each workshop in a declarative syntax, allows the validate.sh to be more broadly applied to other workshops, and allows dependencies to be updated without rebuilding the docker container.

Alternatives

1. Do nothing and continue to update the validate.sh
2. Store the dependency list in another configuration file/container.

Additional Info

S3_BINARIES=(datical_admin.lic DaticalDB-linux.gtk.x86_64-5.2.5347.jar linuxx64_12201_database.zip oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm oracle-instantclient12.2-tools-12.2.0.1.0-1.x86_64.rpm oracle-instantclient12.2-jdbc-12.2.0.1.0-1.x86_64.rpm oracle-instantclient12.2-sqlplus-12.2.0.1.0-1.x86_64.rpm)

daf_tests sometimes fails with panic: interface conversion: error is *url.Error, not *main.AppError

Expected Behavior

daf_tests should complete normally

Actual Behavior

It appears that the daf_tests is not gracefully handling errors when the patients app is not responding

Steps to Reproduce the Problem

Stop the application on testweb and then run daf_tests

Version

Indicate the relevant versions of components. This may include the Titan
package version, operating system version, or other components relevant to
the project.

Additional Information

+ /var/lib/jenkins/daf_tests

2019/11/01 21:01:40 Initializing Resty Client

2019/11/01 21:01:40 Logging in to http://testweb:8080/auth/login

RESTY 2019/11/01 21:01:40 ERROR [Post http://testweb:8080/auth/login: dial tcp 10.0.1.73:8080: connect: connection refused] Attempt [1]

RESTY 2019/11/01 21:01:45 ERROR [Post http://testweb:8080/auth/login: dial tcp 10.0.1.73:8080: connect: connection refused] Attempt [2]

RESTY 2019/11/01 21:01:53 ERROR [Post http://testweb:8080/auth/login: dial tcp 10.0.1.73:8080: connect: connection refused] Attempt [3]

2019/11/01 21:02:05 Fetching all patients

2019/11/01 21:02:05 http://testweb:8080/patients/

RESTY 2019/11/01 21:02:05 ERROR [Get http://testweb:8080/patients/: dial tcp 10.0.1.73:8080: connect: connection refused] Attempt [1]

RESTY 2019/11/01 21:02:10 ERROR [Get http://testweb:8080/patients/: dial tcp 10.0.1.73:8080: connect: connection refused] Attempt [2]

RESTY 2019/11/01 21:02:16 ERROR [Get http://testweb:8080/patients/: dial tcp 10.0.1.73:8080: connect: connection refused] Attempt [3]

panic: interface conversion: error is *url.Error, not *main.AppError



goroutine 1 [running]:

main.main()

	/Users/abowen/Downloads/scripts/daf_tests/main.go:25 +0x1c4f

script returned exit code 2

Build continues if go compile fails

Expected Behavior

If the make action in BINARY_BUILD() fails, the entire build should stop and fail with a non-zero exit code

Actual Behavior

The build continues

Steps to Reproduce the Problem

Type some random characters in any of the *.go files and then build a workshop. You will see errors in the log, but the build will continue.

Version

=< 3.0

Additional Information

go get -d
env CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-X main.version=1.0.5o" -o ./bin/linux64/tw_provision
# _/build/go/tw_provision_oracle
./tw_provision.go:1326:2: undefined: bam
make: *** [Makefile:12: build] Error 2
echo $GOPATH
/go
go get -d
env CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-X main.version=1.0.7p" -o ./bin/linux64/tw_provision
BATCH 1

build not always properly exiting when encountering errors

Expected Behavior

Build should exit with a non-zero exit code when encountering fatal errors. This allows pipeline builds to continue when it should stop them.

Actual Behavior

Build process stops, but returns successfully

Steps to Reproduce the Problem

Start a build and then terminate the AWS instance it is executing packer/ansible against

Version

all

Additional Information

�[1;32m==> delphix-tcw-jumpbox: No volumes to clean up, skipping�[0m
�[1;32m==> delphix-tcw-jumpbox: Deleting temporary security group...�[0m
�[1;32m==> delphix-tcw-jumpbox: Deleting temporary keypair...�[0m
�[1;31mBuild 'delphix-tcw-jumpbox' errored: Error executing Ansible: Non-zero exit status: exit status 2�[0m

==> Some builds didn't complete successfully and had errors:
--> delphix-tcw-jumpbox: Error executing Ansible: Non-zero exit status: exit status 2

==> Builds finished but no artifacts were created.
It took 369 seconds to complete /bin/packer_build.sh
It took 369 seconds to complete /bin/packer_build.sh
Script finished Successfully
mv: can't rename '/build/WAIT.log': No such file or directory

Include the Delphix Jenkins plugin in the TCW workshop

Problem

The tcw environment currently does not include the Delphix Jenkins plugin. Some users might like to explore and play with the Delphix Jenkins plugin

Solution

Install the Delphix Jenkins plugin in the TCW

Alternatives

Don't include it and let people install it themselves.

Additional Info

https://plugins.jenkins.io/delphix

masking city

Expected Behavior

we should NOT be masking the city in the demo- so we can find the records that we add to prod.

Actual Behavior

CIty is getting masked when data refreshes happen

Steps to Reproduce the Problem

add data to prod, refresh from prod, you can't find your silly city name

Version

3.0 RC1