Comments (4)
@mbossenbroek I'm interested in doing this - does PigPen introduce any complications beyond AWS' documented "submit a pig script" process?
If I can get enough hints to make it work I'd be happy to write a step by step for this issue.
from pigpen.
I don't believe it should be any different. You should be able to build an uberjar, generate a script, and run it. Looking forward to see what you come up with!
from pigpen.
Hi @mbossenbroek,
Unfortunately I can't file a PR against the wiki, so I'm unsure how you'd like me to best supply these docs.
I've documented PigPen in an EMR environment for our use here, the guide is completely generic and you're free to use it:
https://github.com/technion/bigdatawanna/tree/master/pigpen
from pigpen.
Thanks! I'll incorporate this into the wiki
from pigpen.
Related Issues (20)
- Strange behavior of count distinct HOT 2
- Add support for distributed cache on the Cascading platform
- Release HOT 2
- Cascading: Optimize co-group with all folds
- Cascading: Add docs & tutorial
- Cascading: Update parquet and avro storage to work with cascading
- CUBE/ROLLUP in PigPen HOT 2
- Libraries/Functions in closures HOT 7
- Should locally executed load functions support compression? HOT 2
- Weird error when used with prismatic plumbing HOT 13
- Use cascading-hadoop2-mr1 by default HOT 10
- allow custom properties to be passed to FlowConnector when creating a flow HOT 2
- Add a pigpen.pig/dump command
- Tutorial error: Pig version 0.12.0-cdh5.4.2,0.14 is right. HOT 5
- Hadoop Versions lists hadoop-client twice in dependencies. HOT 1
- clojure.lang.ExceptionInfo: :auto not supported on headerless data. {} HOT 17
- pigpen.core store functions don't quite work HOT 2
- Incorrect script generation with large number of fields (parquet) HOT 4
- Doc CSS is broken HOT 1
- Is this project being maintained? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pigpen.