googlecloudplatform / df-ml-anomaly-detection Goto Github PK
View Code? Open in Web Editor NEWStreaming Anomaly Detection Solution by using Pub/Sub, Dataflow, BQML & Cloud DLP
License: Apache License 2.0
Streaming Anomaly Detection Solution by using Pub/Sub, Dataflow, BQML & Cloud DLP
License: Apache License 2.0
We currently use model = tf.contrib.estimator.forward_features(model,KEY_COLUMN) for the model creation which not supported for TF 2.0.
Need to upgrade the code for TF 2.0
https://www.tensorflow.org/guide/upgrade
@GoogleCloudPlatform/streaming-analytics-ai
When I followed the official documentation and run the command, I got the error as follows.
gcloud beta dataflow flex-template run "anomaly-detection" \
--project=myProject \
--region=us-central1 \
--template-file-gcs-location=gs://df-ml-anomaly-detection-mock-data/dataflow-flex-template/dynamic_template_finserv_fraud_detection.json \
--parameters=autoscalingAlgorithm="NONE",\
numWorkers=30,\
maxNumWorkers=30,\
workerMachineType=n1-highmem-8,\
subscriberId=projects/myProject/subscriptions/sample_data,\
tableSpec=myProject:fraud_detection.transactions,\
outlierTableSpec=myProject:fraud_detection.fraud_prediction,\
inputFilePattern=gs://df-ml-anomaly-detection-mock-data/finserv_fraud_detection/fraud_data_kaggle.json,\
modelId=fraud_detection_with_key,\
versionId=v1,\
keyRange=1024,\
batchSize=500000
Error message from logging;
docker: Error response from daemon: pull access denied for gcr.io/cloud-solutions-images/df-ml-finserv-anomaly-detection, repository does not exist or may require 'docker login': denied: Permission denied for "latest" from request "/v2/cloud-solutions-images/df-ml-finserv-anomaly-detection/manifests/latest".
Here is logging output.
https://cloud.google.com/architecture/detecting-anomalies-in-financial-transactions
In the tutorial, "gcloud beta dataflow flex-template run " is used to run the dataflow pipeline. But, the template "gs://df-ml-anomaly-detection-mock-data/dataflow-flex-template/dynamic_template_finserv_fraud_detection.json" is not available.
How can we build a flex-template or run the dataflow pipeline for Anomaly Detection Reference Architecture Using Cloud AI.
Upgrade to 2.19 breaks Row handler as the KV is no longer available.
https://issues.apache.org/jira/browse/BEAM-7116
I cant find the containers ie. gcr.io/cloud-solutions-images/df-data-generator
is there a new repo or clone somewhere?
This issue was automatically created by Allstar.
Security Policy Violation
Project is out of compliance with Binary Artifacts policy: binaries present in source code
Rule Description
Binary Artifacts are an increased security risk in your repository. Binary artifacts cannot be reviewed, allowing the introduction of possibly obsolete or maliciously subverted executables. For more information see the Security Scorecards Documentation for Binary Artifacts.
Remediation Steps
To remediate, remove the generated executable artifacts from the repository.
Artifacts Found
Additional Information
This policy is drawn from Security Scorecards, which is a tool that scores a project's adherence to security best practices. You may wish to run a Scorecards scan directly on this repository for more details.
Allstar has been installed on all Google managed GitHub orgs. Policies are gradually being rolled out and enforced by the GOSST and OSPO teams. Learn more at http://go/allstar
This issue will auto resolve when the policy is in compliance.
Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.