Coder Social home page Coder Social logo

slalom / dataops-docs Goto Github PK

View Code? Open in Web Editor NEW
3.0 3.0 88.0 18.62 MB

Quickly get started with DevOps tools and best practices for building modern data solutions.

Home Page: https://docs.dataops.tk

License: MIT License

Dockerfile 0.93% Batchfile 6.16% Shell 0.18% HTML 3.63% Jupyter Notebook 89.10%

dataops-docs's Introduction

Welcome to the dataops docs site!

The tools and documentation provided here are designed to accelerate data and analytics projects for the Slalom Team and our clients.

Guides and Documentation

Labs

See the Labs Page for all of the Slalom DataOps Labs.

Other Important Links

  • dataops-tools - The primary DataOps git repo for tools mentioned here in these docs.
  • dataops-docs - Link to this documentation repo. (Submit documentation bugs and enhancement request here!)
  • dataops-infra - Link to the Infrastructure Catalog repo.
  • docs.dataops.tk/backlog - DataOps backlog and current development status.

About Slalom

Slalom is a modern consulting firm focused on strategy, technology, and business transformation. If you'd like to learn more about how we can help accelerate your business transformation and deliver best-in-class DataOps solutions, please visit us on Slalom.com, reach out to your local Slalom representative, or ping any one of our many Slalom open source software contributors here on GitHub.

dataops-docs's People

Contributors

aaronsteers avatar achuul8er avatar aditomassi avatar alex-b-griffin avatar alexbcook avatar alialsadadi avatar aliasghararabi avatar andrewgarfinkel avatar atopp-slalom avatar bbainsslalom avatar biancaorozco avatar brandiskanes avatar cmgonzalezjr avatar david-holtkamp avatar domedwards avatar egelnoteagle avatar etishtalreja140 avatar farrizon-slalom avatar ihafidh avatar jerry-mathew avatar kevincifone avatar madhurima-nath avatar maripqz avatar michalsicak avatar mithra-ramesh avatar nathaniellarson avatar park-peter avatar peter-griffin-slalom avatar vamsi-kurakula avatar zariffic avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

dataops-docs's Issues

TODO: An AWS Glue Lab

As an instructor, I'd like to have a glue lab that I can use to quickly get people up and running with glue transform jobs.

TODO: Reusable framework for naming conventions

As an architect starting a new project, I'd like to have a turnkey manner of establishing naming conventions. This should come out of box with a set of modern/suggested conventions, with the ability to customize or "opt out" of certain settings per each individual environment.

TODO: "Kitchen Sink" AWS Lab

As trainers, we would like to have a "kitchen sink" lab that contains a ton of things for people to play with, while requiring very little advanced knowledge and little or no secrets management (aside from AWS Creds setup).

Unable to install PowerBI via chocolaty link

Hope this is the right log... the command line said something about the hash not matching before it exited.

2020-08-29 17:43:38,862 13376 [DEBUG] - ---------------------------Script Execution---------------------------
2020-08-29 17:43:38,863 13376 [DEBUG] - Running 'ChocolateyScriptRunner' for PowerBI v2.84.802.0 with packageScript 'C:\ProgramData\chocolatey\lib\PowerBI\tools\ChocolateyInstall.ps1', packageFolder:'C:\ProgramData\chocolatey\lib\PowerBI', installArguments: '', packageParameters: '',
2020-08-29 17:43:38,867 13376 [DEBUG] - Running 'C:\ProgramData\chocolatey\lib\PowerBI\tools\ChocolateyInstall.ps1'
2020-08-29 17:43:38,903 13376 [DEBUG] - Running Install-ChocolateyPackage -url 'https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup.exe' -url64bit 'https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup_x64.exe' -checksum '423139def3dbbdf0f5682fdd88921eaface84ff48ef2e2a8d20e4341ca4a4364' -checksum64 '7f622cc80248a826c57667a72318b4694e7e22c1b4ad1a2ad815e70fd599adcf' -packageName 'PowerBI' -checksumType 'SHA256' -fileType 'EXE' -validExitCodes '0 3010 1641' -silentArgs '-quiet -norestart ACCEPT_EULA=1 -log "C:\Users\Peter Griffin\AppData\Local\Temp\chocolatey\PowerBI.2.84.802.0.Install.log"'
2020-08-29 17:43:38,926 13376 [DEBUG] - Running Get-ChocolateyWebFile -packageName 'PowerBI' -fileFullPath 'C:\Users\Peter Griffin\AppData\Local\Temp\chocolatey\PowerBI\2.84.802.0\PowerBIInstall.EXE' -url 'https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup.exe' -url64bit 'https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup_x64.exe' -checksum '423139def3dbbdf0f5682fdd88921eaface84ff48ef2e2a8d20e4341ca4a4364' -checksumType 'SHA256' -checksum64 '7f622cc80248a826c57667a72318b4694e7e22c1b4ad1a2ad815e70fd599adcf' -checksumType64 '' -options 'System.Collections.Hashtable' -getOriginalFileName 'True'
2020-08-29 17:43:38,934 13376 [DEBUG] - Running Get-ProcessorBits -compare '64'
2020-08-29 17:43:38,943 13376 [DEBUG] - CPU is 64 bit
2020-08-29 17:43:38,949 13376 [DEBUG] - Setting url to 'https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup_x64.exe' and bitPackage to 64
2020-08-29 17:43:38,969 13376 [DEBUG] - Running Get-WebFileName -url 'https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup_x64.exe' -defaultName 'PowerBIInstall.EXE'
2020-08-29 17:43:39,109 13376 [DEBUG] - Using response url to determine file name. 'https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup_x64.exe'
2020-08-29 17:43:39,111 13376 [DEBUG] - File name determined from url is 'PBIDesktopSetup_x64.exe'
2020-08-29 17:43:39,118 13376 [DEBUG] - Running Get-WebHeaders -url 'https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup_x64.exe' -ErrorAction 'Stop'
2020-08-29 17:43:39,119 13376 [DEBUG] - Setting the UserAgent to 'chocolatey command line'
2020-08-29 17:43:39,120 13376 [DEBUG] - Request Headers:
2020-08-29 17:43:39,122 13376 [DEBUG] - 'Accept':'/'
2020-08-29 17:43:39,123 13376 [DEBUG] - 'User-Agent':'chocolatey command line'
2020-08-29 17:43:39,187 13376 [DEBUG] - Response Headers:
2020-08-29 17:43:39,189 13376 [DEBUG] - 'x-ms-blob-content-md5':'gxBjMZEV7t//uLXPsvfRJA=='
2020-08-29 17:43:39,190 13376 [DEBUG] - 'Connection':'keep-alive'
2020-08-29 17:43:39,190 13376 [DEBUG] - 'Accept-Ranges':'bytes'
2020-08-29 17:43:39,191 13376 [DEBUG] - 'Content-Length':'306132560'
2020-08-29 17:43:39,191 13376 [DEBUG] - 'Content-Type':'application/octet-stream'
2020-08-29 17:43:39,192 13376 [DEBUG] - 'Date':'Sun, 30 Aug 2020 00:43:39 GMT'
2020-08-29 17:43:39,192 13376 [DEBUG] - 'ETag':'"0x8D84B7F41E87161"'
2020-08-29 17:43:39,193 13376 [DEBUG] - 'Last-Modified':'Fri, 28 Aug 2020 18:22:01 GMT'
2020-08-29 17:43:39,193 13376 [DEBUG] - 'Server':'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0'
2020-08-29 17:43:39,200 13376 [INFO ] - File appears to be downloaded already. Verifying with package checksum to determine if it needs to be redownloaded.
2020-08-29 17:43:39,206 13376 [DEBUG] - Running Get-ChecksumValid -file 'C:\Users\Peter Griffin\AppData\Local\Temp\chocolatey\PowerBI\2.84.802.0\PBIDesktopSetup_x64.exe' -checksum '7f622cc80248a826c57667a72318b4694e7e22c1b4ad1a2ad815e70fd599adcf' -checksumType 'SHA256' -originalUrl 'https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup_x64.exe' -ErrorAction 'Stop'
2020-08-29 17:43:39,209 13376 [DEBUG] - checksum.exe found at 'C:\ProgramData\chocolatey\helpers..\tools\checksum.exe'
2020-08-29 17:43:39,210 13376 [DEBUG] - Executing command ['C:\ProgramData\chocolatey\helpers..\tools\checksum.exe' -c="7f622cc80248a826c57667a72318b4694e7e22c1b4ad1a2ad815e70fd599adcf" -t="SHA256" -f="C:\Users\Peter Griffin\AppData\Local\Temp\chocolatey\PowerBI\2.84.802.0\PBIDesktopSetup_x64.exe"]
2020-08-29 17:43:40,553 13376 [DEBUG] - Command ['C:\ProgramData\chocolatey\helpers..\tools\checksum.exe' -c="7f622cc80248a826c57667a72318b4694e7e22c1b4ad1a2ad815e70fd599adcf" -t="SHA256" -f="C:\Users\Peter Griffin\AppData\Local\Temp\chocolatey\PowerBI\2.84.802.0\PBIDesktopSetup_x64.exe"] exited with '1'.
2020-08-29 17:43:40,556 13376 [DEBUG] - Existing file failed checksum. Will be redownloaded from url.
2020-08-29 17:43:40,557 13376 [INFO ] - Downloading PowerBI 64 bit
from 'https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup_x64.exe'
2020-08-29 17:43:40,566 13376 [DEBUG] - Running Get-WebFile -url 'https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup_x64.exe' -fileName 'C:\Users\Peter Griffin\AppData\Local\Temp\chocolatey\PowerBI\2.84.802.0\PBIDesktopSetup_x64.exe' -options 'System.Collections.Hashtable'
2020-08-29 17:43:40,567 13376 [DEBUG] - Setting request timeout to 30000
2020-08-29 17:43:40,568 13376 [DEBUG] - Setting read/write timeout to 2700000
2020-08-29 17:43:40,569 13376 [DEBUG] - Setting the UserAgent to 'chocolatey command line'
2020-08-29 17:43:40,701 13376 [DEBUG] - Downloading https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup_x64.exe to C:\Users\Peter Griffin\AppData\Local\Temp\chocolatey\PowerBI\2.84.802.0\PBIDesktopSetup_x64.exe
2020-08-29 17:44:10,744 13376 [INFO ] -
2020-08-29 17:44:10,745 13376 [INFO ] - Download of PBIDesktopSetup_x64.exe (291.95 MB) completed.
2020-08-29 17:44:13,792 13376 [DEBUG] - No runtime virus checking built into FOSS Chocolatey. Check out Pro/Business - https://chocolatey.org/compare
2020-08-29 17:44:13,795 13376 [DEBUG] - Verifying package provided checksum of '7f622cc80248a826c57667a72318b4694e7e22c1b4ad1a2ad815e70fd599adcf' for 'C:\Users\Peter Griffin\AppData\Local\Temp\chocolatey\PowerBI\2.84.802.0\PBIDesktopSetup_x64.exe'.
2020-08-29 17:44:13,796 13376 [DEBUG] - Running Get-ChecksumValid -file 'C:\Users\Peter Griffin\AppData\Local\Temp\chocolatey\PowerBI\2.84.802.0\PBIDesktopSetup_x64.exe' -checksum '7f622cc80248a826c57667a72318b4694e7e22c1b4ad1a2ad815e70fd599adcf' -checksumType 'SHA256' -originalUrl 'https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup_x64.exe'
2020-08-29 17:44:13,796 13376 [DEBUG] - checksum.exe found at 'C:\ProgramData\chocolatey\helpers..\tools\checksum.exe'
2020-08-29 17:44:13,797 13376 [DEBUG] - Executing command ['C:\ProgramData\chocolatey\helpers..\tools\checksum.exe' -c="7f622cc80248a826c57667a72318b4694e7e22c1b4ad1a2ad815e70fd599adcf" -t="SHA256" -f="C:\Users\Peter Griffin\AppData\Local\Temp\chocolatey\PowerBI\2.84.802.0\PBIDesktopSetup_x64.exe"]
2020-08-29 17:44:15,113 13376 [DEBUG] - Command ['C:\ProgramData\chocolatey\helpers..\tools\checksum.exe' -c="7f622cc80248a826c57667a72318b4694e7e22c1b4ad1a2ad815e70fd599adcf" -t="SHA256" -f="C:\Users\Peter Griffin\AppData\Local\Temp\chocolatey\PowerBI\2.84.802.0\PBIDesktopSetup_x64.exe"] exited with '1'.
2020-08-29 17:44:15,117 13376 [ERROR] - ERROR: Checksum for 'C:\Users\Peter Griffin\AppData\Local\Temp\chocolatey\PowerBI\2.84.802.0\PBIDesktopSetup_x64.exe' did not meet '7f622cc80248a826c57667a72318b4694e7e22c1b4ad1a2ad815e70fd599adcf' for checksum type 'SHA256'. Consider passing the actual checksums through with --checksum --checksum64 once you validate the checksums are appropriate. A less secure option is to pass --ignore-checksums if necessary.
2020-08-29 17:44:15,119 13376 [DEBUG] - Built-in PowerShell host called with ['[System.Threading.Thread]::CurrentThread.CurrentCulture = '';[System.Threading.Thread]::CurrentThread.CurrentUICulture = ''; & import-module -name 'C:\ProgramData\chocolatey\helpers\chocolateyInstaller.psm1'; & 'C:\ProgramData\chocolatey\helpers\chocolateyScriptRunner.ps1' -packageScript 'C:\ProgramData\chocolatey\lib\PowerBI\tools\ChocolateyInstall.ps1' -installArguments '' -packageParameters '''] exited with '-1'.
2020-08-29 17:44:15,121 13376 [DEBUG] - Calling command ['"C:\WINDOWS\System32\shutdown.exe" /a']
2020-08-29 17:44:15,149 13376 [DEBUG] - Command ['"C:\WINDOWS\System32\shutdown.exe" /a'] exited with '1116'
2020-08-29 17:44:15,221 13376 [DEBUG] - Capturing package files in 'C:\ProgramData\chocolatey\lib\PowerBI'
2020-08-29 17:44:15,222 13376 [DEBUG] - Found 'C:\ProgramData\chocolatey\lib\PowerBI\PowerBI.nupkg'
with checksum '44BC02B9B3012D30247F8EE2138910F4'
2020-08-29 17:44:15,223 13376 [DEBUG] - Found 'C:\ProgramData\chocolatey\lib\PowerBI\PowerBI.nuspec'
with checksum '64C9338FDE3153CDC676A59BF39442B3'
2020-08-29 17:44:15,224 13376 [DEBUG] - Found 'C:\ProgramData\chocolatey\lib\PowerBI\tools\ChocolateyInstall.ps1'
with checksum '67FB4D8D42F7784B727F81D7E9EA5C60'
2020-08-29 17:44:15,233 13376 [DEBUG] - Attempting to delete file "C:\ProgramData\chocolatey.chocolatey\PowerBI.2.84.802.0.arguments".
2020-08-29 17:44:15,233 13376 [DEBUG] - Attempting to delete file "C:\ProgramData\chocolatey.chocolatey\PowerBI.2.84.802.0.extra".
2020-08-29 17:44:15,234 13376 [DEBUG] - Attempting to delete file "C:\ProgramData\chocolatey.chocolatey\PowerBI.2.84.802.0.version".
2020-08-29 17:44:15,234 13376 [DEBUG] - Attempting to delete file "C:\ProgramData\chocolatey.chocolatey\PowerBI.2.84.802.0.sxs".
2020-08-29 17:44:15,235 13376 [DEBUG] - Attempting to delete file "C:\ProgramData\chocolatey.chocolatey\PowerBI.2.84.802.0.pin".
2020-08-29 17:44:15,237 13376 [DEBUG] - Attempting to delete directory "C:\ProgramData\chocolatey\lib-bad\PowerBI".
2020-08-29 17:44:15,240 13376 [DEBUG] - Sending message 'HandlePackageResultCompletedMessage' out if there are subscribers...
2020-08-29 17:44:15,242 13376 [ERROR] - The install of powerbi was NOT successful.
2020-08-29 17:44:15,243 13376 [ERROR] - Error while running 'C:\ProgramData\chocolatey\lib\PowerBI\tools\ChocolateyInstall.ps1'.
See log for details.
2020-08-29 17:44:15,245 13376 [DEBUG] - Moving 'C:\ProgramData\chocolatey\lib\PowerBI'
to 'C:\ProgramData\chocolatey\lib-bad\PowerBI'
2020-08-29 17:44:17,248 13376 [DEBUG] - Attempting to delete file "C:\Users\Peter Griffin\AppData\Local\NuGet\Cache\PowerBI.2.84.802.0.nupkg".
2020-08-29 17:44:17,254 13376 [WARN ] -
Chocolatey installed 0/1 packages. 1 packages failed.
See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
2020-08-29 17:44:17,255 13376 [INFO ] -
2020-08-29 17:44:17,255 13376 [ERROR] - Failures
2020-08-29 17:44:17,256 13376 [ERROR] - - powerbi (exited -1) - Error while running 'C:\ProgramData\chocolatey\lib\PowerBI\tools\ChocolateyInstall.ps1'.
See log for details.
2020-08-29 17:44:17,258 13376 [DEBUG] - Sending message 'PostRunMessage' out if there are subscribers...
2020-08-29 17:44:17,267 13376 [DEBUG] - Exiting with -1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.