Comments (6)
@drcat101 Any suggestions?
from building-machine-learning-pipelines.
Resolved by adding ipython extra package:
setup_options.extra_packages = [ './tensorflow_data_validation-0.23.0-cp37-cp37m-manylinux2010_x86_64.whl', 'ipython-7.17.0-py3-none-any.whl']
from building-machine-learning-pipelines.
@mshearer0 thank you for adding the solution.
from building-machine-learning-pipelines.
@mshearer0 Can you please share your statsgen
setup? Are you visualizing the results from the components after the Dataflow Runner completes the execution?
from building-machine-learning-pipelines.
@hanneshapke as per Chapter4:
import tensorflow_data_validation as tfdv
from apache_beam.options.pipeline_options import (
PipelineOptions, GoogleCloudOptions, StandardOptions)
options = PipelineOptions()
google_cloud_options = options.view_as(GoogleCloudOptions)
google_cloud_options.project = 'xxx'
google_cloud_options.job_name = 'beamtfdv'
google_cloud_options.staging_location = 'gs://xxx/staging'
google_cloud_options.temp_location = 'gs://xxx/tmp'
google_cloud_options.region='europe-west1'
options.view_as(StandardOptions).runner = 'DataflowRunner'
from apache_beam.options.pipeline_options import SetupOptions
setup_options = options.view_as(SetupOptions)
setup_options.extra_packages = [ 'tensorflow_data_validation-0.23.0-cp37-cp37m-manylinux2010_x86_64.whl', 'ipython-7.18.1-py3-none-any.whl']
data_set_path = 'gs://xxx/consumer-complaints.tfrecords'
output_path = 'gs://xxx/'
tfdv.generate_statistics_from_tfrecord(data_set_path,
output_path=output_path,
pipeline_options=options)
from building-machine-learning-pipelines.
Thank you @mshearer0 . I wonder if iPython was previously pre-installed in the dataflow instances. I am glad the setup is working for you now.
from building-machine-learning-pipelines.
Related Issues (20)
- Makefile `install-deps` fails HOT 1
- How to read JSON files with FileBasedExampleGen HOT 1
- Data Preprocessing - Handling Multilabel Data HOT 1
- Why Infra Validator Component is not even covered? HOT 4
- The way to extend BaseExampleGenExecutor has changed in version 0.23 HOT 1
- GOOGLE_APPLICATION_CREDENTIALS HOT 1
- Unable to install requirements HOT 14
- Can't run Kubeflow sample code HOT 4
- context.run(statistics_gen) interactive pipeline throwing error HOT 1
- Do we need to create schema for train dataset or entire dataset to compare with eval and test dataset? HOT 1
- Modification for `pre-experiment-pipeline/experiment_6Mar.ipynb` HOT 1
- is it possible to get TFserving tutorial for the existing pipeline and for the same data?
- Dockerfile for creating image used to Train HOT 2
- tfx.utils.dsl_utils isn't supported anymore and raises errors. HOT 1
- Interactive Pipeline crashes on different stages HOT 2
- Chapter 2 example error HOT 1
- There issues installing the requirements.txt in ubutu 22.04 HOT 1
- Issue installing requirements HOT 1
- problem while serving with docker container
- Examples pass to evaluator component
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from building-machine-learning-pipelines.