Include Snowflake staged file metadata in LoadFileOperator about astro-sdk

[phanikumv]

@pankajkoti adding it to your backlog

[pankajkoti]

Okay @phanikumv 👍🏾

[pankajkoti]

hi @Andrew-Wichmann

We have just released Astro SDK 1.7.0 https://pypi.org/project/astro-sdk-python/1.7.0/ which includes the PR #2023 supporting the implementation for this issue.

Would you like to try out? Here is an example operator usage

astro-sdk/python-sdk/example_dags/example_load_file.py

Lines 392 to 411 in 7583814

	aql.load_file(
	task_id="s3_to_snowflake_native_with_metadata_columns",
	input_file=File("s3://astro-sdk/python_sdk/example_dags/data/sample.csv", conn_id=AWS_CONN_ID),
	output_table=Table(
	conn_id=SNOWFLAKE_CONN_ID,
	),
	load_options=[
	SnowflakeLoadOptions(
	file_options={"SKIP_HEADER": 1, "SKIP_BLANK_LINES": True},
	copy_options={"ON_ERROR": "CONTINUE"},
	metadata_columns=[
	"METADATA$FILENAME",
	"METADATA$FILE_ROW_NUMBER",
	"METADATA$FILE_CONTENT_KEY",
	"METADATA$FILE_LAST_MODIFIED",
	"METADATA$START_SCAN_TIME",
	],
	)
	],
	)

[Andrew-Wichmann]

Nice thank you!