Thiết kế website giá rẻ

Question

I am trying to install airflow for the first time on my local machine. As part of the process I have installed :
python 3.12.4
VSCode for executing python commands and downloading necessary packages for airflow.

As part of installation steps I have executed below commands:

mkdir airflow_project

cd airflow_project

python -m venv venv_airflow

.venv_airflowScriptsActivate

pip install --upgrade pip

I dont see airflow.cfg in the airflow folder, so I copied it from config_templates folder into airflow folder as airflow.cfg and edited it. Is that right?

Edited the airflow.cfg and replaced the values for line:
sql_alchemy_conn = snowflake://:@/?warehouse=&schema=&role=

I couldn’t find airflow.cfg in the airflow folder, so I copied it from config_templates folder into airflow folder as airflow.cfg and edited it. Is that right?

airflow db init

(venv_aaairflow4) PS C:UsersXXXXXXXproj_airflowvenv_aaairflow4Libsite-packagesairflow> airflow db migrate
C:UsersXXXXXXXproj_airflowvenv_aaairflow4Libsite-packagesairflow__init__.py:36: RuntimeWarning: Airflow currently can be run on POSIX-compliant Operating Systems. For development, it is regularly tested on fairly modern Linux Distros and recent versions of macOS. On Windows you can run it via WSL2 (Windows Subsystem for Linux 2) or via Linux Containers. The work to add Windows support is tracked via https://github.com/apache/airflow/issues/10388, but it is not a high priority.
  warnings.warn(
OSError while attempting to symlink the latest log directory
[2024-06-12T15:21:10.847+0100] {connection.py:399} INFO - Snowflake Connector for Python Version: 3.10.1, Python Version: 3.12.4, Platform: Windows-10-10.0.19043-SP0
[2024-06-12T15:21:10.850+0100] {connection.py:1239} INFO - This connection is in OCSP Fail Open Mode. TLS Certificates would be checked for validity and revocation status. Any other Certificate Revocation related exceptions or OCSP Responder failures would be disregarded in favor of connectivity.
[2024-06-12T15:21:11.559+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:11.617+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:11.727+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:11.765+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:11.766+0100] {cli_action_loggers.py:137} WARNING - Failed to log action (snowflake.connector.errors.ProgrammingError) 002003 (42S02): 01b4f61d-0000-9962-0000-6ee900017106: SQL compilation error:
Table 'LOG' does not exist or not authorized.
[SQL: INSERT INTO log (dttm, event, owner, extra) VALUES (%(dttm)s, %(event)s, %(owner)s, %(extra)s)]
[parameters: {'dttm': datetime.datetime(2024, 6, 12, 14, 21, 10, 847011, tzinfo=Timezone('UTC')), 'event': 'cli_migratedb', 'owner': 'XXXXXXX', 'extra': '{"host_name": "LAP11171", "full_command": "['C:\\\\Users\\\\XXXXXXX\\\\proj_airflow\\\\venv_aaairflow4\\\\Scripts\\\\airflow', 'db', 'migrate']"}'}]
(Background on this error at: https://sqlalche.me/e/14/f405)
DB: snowflake://AIRFLOWADMIN:***@icfnm-ld92/AIRFLOW?role=RL_AIRFLOW_ADMIN&schema=SCHAIRFLOW&warehouse=WHSTDXSAIRFLOW
Performing upgrade to the metadata database snowflake://AIRFLOWADMIN:***@icfnm-ld092/AIRFLOW?role=RL_AIRFLOW_ADMIN&schema=SCHAIRFLOW&warehouse=WHSTDXSAIRFLOW
[2024-06-12T15:21:12.019+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.130+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.193+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.254+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.332+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.382+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.419+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.455+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.495+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.547+0100] {cursor.py:1149} INFO - Number of results in first chunk: 0
[2024-06-12T15:21:12.584+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.621+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.657+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.698+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.743+0100] {cursor.py:1149} INFO - Number of results in first chunk: 0
[2024-06-12T15:21:12.792+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.838+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.874+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.912+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:12.962+0100] {cursor.py:1149} INFO - Number of results in first chunk: 0
[2024-06-12T15:21:12.999+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.034+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.076+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.114+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.168+0100] {cursor.py:1149} INFO - Number of results in first chunk: 0
[2024-06-12T15:21:13.207+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.242+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.277+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.317+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.361+0100] {cursor.py:1149} INFO - Number of results in first chunk: 0
[2024-06-12T15:21:13.400+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.440+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.478+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.516+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.574+0100] {cursor.py:1149} INFO - Number of results in first chunk: 0
[2024-06-12T15:21:13.614+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.651+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.690+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.731+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.788+0100] {cursor.py:1149} INFO - Number of results in first chunk: 0
[2024-06-12T15:21:13.834+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.874+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.920+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:13.969+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:14.016+0100] {cursor.py:1149} INFO - Number of results in first chunk: 0
[2024-06-12T15:21:14.053+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:14.093+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:14.187+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:14.224+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:14.263+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:14.343+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:14.382+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:14.419+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:14.459+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
[2024-06-12T15:21:14.497+0100] {cursor.py:1149} INFO - Number of results in first chunk: 1
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:UsersXXXXXXXproj_airflowvenv_aaairflow4Scriptsairflow.exe__main__.py", line 7, in <module>
  File "C:UsersXXXXXXXproj_airflowvenv_aaairflow4Libsite-packagesairflow__main__.py", line 58, in main
    args.func(args)
  File "C:UsersXXXXXXXproj_airflowvenv_aaairflow4Libsite-packagesairflowclicli_config.py", line 49, in command
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:UsersXXXXXXXproj_airflowvenv_aaairflow4Libsite-packagesairflowutilscli.py", line 114, in wrapper
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "C:UsersXXXXXXXproj_airflowvenv_aaairflow4Libsite-packagesairflowutilsproviders_configuration_loader.py", line 55, in wrapped_function
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:UsersXXXXXXXproj_airflowvenv_aaairflow4Libsite-packagesairflowclicommandsdb_command.py", line 130, in migratedb
    db.upgradedb(
  File "C:UsersXXXXXXXproj_airflowvenv_aaairflow4Libsite-packagesairflowutilssession.py", line 79, in wrapper
    return func(*args, session=session, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:UsersXXXXXXXproj_airflowvenv_aaairflow4Libsite-packagesairflowutilsdb.py", line 1617, in upgradedb
    if not to_revision and not _get_current_revision(session=session) and not use_migration_files:
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:UsersXXXXXXXproj_airflowvenv_aaairflow4Libsite-packagesairflowutilsdb.py", line 790, in _get_current_revision
    migration_ctx = MigrationContext.configure(conn)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:UsersXXXXXXXproj_airflowvenv_aaairflow4Libsite-packagesalembicruntimemigration.py", line 280, in configure
    return MigrationContext(dialect, connection, opts, environment_context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:UsersXXXXXXXproj_airflowvenv_aaairflow4Libsite-packagesalembicruntimemigration.py", line 208, in __init__
    self.impl = ddl.DefaultImpl.get_by_dialect(dialect)(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:UsersXXXXXXXproj_airflowvenv_aaairflow4Libsite-packagesalembicddlimpl.py", line 133, in get_by_dialect
    return _impls[dialect.name]
           ~~~~~~^^^^^^^^^^^^^^
KeyError: 'snowflake'
[2024-06-12T15:21:14.505+0100] {connection.py:762} INFO - closed
[2024-06-12T15:21:14.506+0100] {connection.py:768} INFO - No async queries seem to be running, deleting session

I am able connect to snowflake cloud using a python script outside virtual environment. So I am sure my username password and other parameters to connect snowflake, are correct. But when I use the same parameters in the airflow.cfg it starts throwing errors.

I tried installing several supporting packages but they didnt work. Ended up getting the same error.
Can you help me resolving the issue?

Thiết kế website giá rẻ

Danh mục

airflow setup on windows