google / orchestra Goto Github PK

When trying to use the new google_analytics operators, the documentation shows that the parameter storage_name_object is templated, however when I pass the following:

import_audience = GoogleAnalyticsDataImportUploadOperator(
        task_id='import_audience_list_{}'.format(destination_info['account_name']),
        storage_bucket=walden_config['result_bucket_name'],
        storage_name_object="audience-{{ds_nodash}}.csv",
        account_id=destination_info['account_id'],
        web_property_id=destination_info['web_property_id'],
        custom_data_source_id=destination_info['custom_data_source_id'],
        mime_type='application/octet-stream',
        api_version='v3',
        api_name='analytics',
        gcp_conn_id='google_cloud_default',
        data_import_filename='audience-{{ds_nodash}}.csv',
        dag=dag
    )

The dag fails with the following error:

HttpError 404 when requesting https://storage.googleapis.com/storage/v1/b/ <my_bucket> /o/audience-%7B%7Bds_nodash%7D%7D.csv?alt=media returned "Not Found">

I've removed my bucket name here, but it is correct in the url

It looks like it's not recognizing the templated {{ds_nodash}} here.

We are running v1.10.2-composer

Hello

My team created a new operators for Airflow that copy files from and to GCS and FTPS.
apache/airflow#6393
apache/airflow#6366

These operators can be replaced by an operator that has been implemented by your team.
https://github.com/google/orchestra/blob/Orchestra-2.0/orchestra/google/cloud/operators/gcp_gcs_operator.py
I think that it makes no sense to duplicate operators and it is better to use the operator that will be provided with the new versions of Airflow/Cloud Composer.

Best regards,

CC: @TobKed

Apache has updated the following function to not use attr.

The following argument is used in Google Marketing Operator

I find some error when trying to use both version of Airflow, especially when dealing with clustered table. In 1.10.3 it was just fine, however it was failed when I'm using 1.10.1.

Hi,

I set up a process which uses the GoogleDisplayVideo360SDFToBigQueryOperator from display_video_360. However for some advertisers which contain a lot of data I got this error.

ERROR - <HttpError 500 when requesting https://www.googleapis.com/doubleclickbidmanager/v1/sdf/download?alt=json returned "Requested Resource Too Large to Return">

Is there anyway that I can reduce the amount of data from this SDF API call ?

Thanks !

When running a DAG on Airflow 2.1 on a VM, the spindle DAG (create_report) is returning an attribute error in the code:
AttributeError: 'Resource' object has no attribute 'queries'
I am not understanding what is the reason, since on composer it seemed to have worked fine.