apache_airflow_on_eks's People
Forkers
christinaet technicsntricks ebertbm mouftizo avi202020 rgolovnya krishna-medikeri gabrielamazonas andraskelle zsolt-torok umair-abro zkan godelian jiajie999 hlopes-bi4all debodirno ecogit-stage avadhesh-mastek atif8tedapache_airflow_on_eks's Issues
terraform apply failing
I ran into an issue while trying out this repository.
the issue occurred during the "terraform apply" step.
Error: Provider produced inconsistent final plan
When expanding the plan for
module.eks_cluster.null_resource.wait_for_cluster[0] to include new values
learned so far during apply, provider "registry.terraform.io/hashicorp/null"
changed the planned action from CreateThenDelete to DeleteThenCreate.
This is a bug in the provider, which should be reported in the provider's own
issue tracker.
it happens when terraform tries to create the aws_launch_configuration for both worker.
Because of the error both neither of the workers are created.
These are the variables that I added:
variable "project_name" {
description = "Name for the test project"
default = "test-project"
type = string
}
variable "environment" {
default = "dev"
}
variable "aws_profile" {
default = "default"
}
variable "aws_region" {
default = "us-east-2"
}
// This will be the ip that will be allowed to connect to Airflow and to the monitoring services by default
variable "allowed_ip" {
default = "0.0.0.0/0"
}
variable "nodes_instance_type_1" {
default = "t3.medium"
type = string
}
variable "nodes_instance_type_2" {
default = "t3.large"
type = string
}
// This RDS Postgres is used by Airflow to store metadata
variable "postgres_user" {
default = "name"
type = string
}
variable "postgres_password" {
default = "XXXXXXX"
type = string
}
And this is the terraform version I am using:
Terraform v0.13.2
+ provider registry.terraform.io/hashicorp/aws v3.5.0
+ provider registry.terraform.io/hashicorp/kubernetes v1.13.1
+ provider registry.terraform.io/hashicorp/local v1.4.0
+ provider registry.terraform.io/hashicorp/null v2.1.2
+ provider registry.terraform.io/hashicorp/random v2.3.0
+ provider registry.terraform.io/hashicorp/template v2.1.2
Airflow depployment failing on EKS Fargate
I have my EKS Fargate cluster with no EC2 as node group. When deployed airflow in EKS using helm the deployment is failing and the pods are terminating and restarting continuously. But the same docker image when taken to EKS with Ec2 airflow is running without issues. Please help me on this.
Connect external repo as a dag folder
First off, thanks for the heavy lifting here!
Whenever there's any change to the Airflow Dags, the Dockerfile inside the dag folder can be built into an image, and pushed into the ECR repository.
Would it be possible to reference a git repository where our Airflow dags are located separately? As Airflow dags usually change more frequently it would be great to have less deployments.
Thanks in advance!
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.