feat: disable sync bucket#102
Merged
Merged
Conversation
Piotr Sierkin (Santhin)
changed the title
Piotr Sierkin (Santhin)
changed the title
Piotr Pękala (p-pekala)
requested changes
May 10, 2023
| @click.option( | ||
| "--sync-bucket", | ||
| is_flag=True, | ||
| default=False, |
Collaborator
There was a problem hiding this comment.
For backward compatibility it should be True by default, isn't it?
Contributor
Author
There was a problem hiding this comment.
The question at hand is whether or not to take UX into account. If a user desires to activate "sync bucket," no action is required. However, if the user wants to deactivate it, they must explicitly provide it as an argument. From my perspective, this process can be somewhat confusing.
Collaborator
There was a problem hiding this comment.
true, but still more confusing will be situation when after upgrading to new version everything stops working. I would assume that sync is by default on as deploy is mostly about this. So maybe instead let's call this flag --disable-bucket-sync ?
Contributor
Author
There was a problem hiding this comment.
I will push an update
Piotr Sierkin (Santhin)
changed the title
Piotr Pękala (p-pekala)
approved these changes
May 12, 2023
Paweł Jurkiewicz (paszynski)
approved these changes
May 12, 2023
Piotr Sierkin (Santhin)
added a commit
that referenced
this pull request
May 15, 2023
* Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * feat: looker integration for compile and deploy, new command: bi * tests fix, looker git with ssh * chore: organizing imports, removed not used variables * chore: dbt2looker as extra dependency, mypy errors fix, bi-git-key-path option instead of key-path * chore: formatting, types-requests dependency for mypy * refactor: refactoring bi, removing bi command, fix tests, moving bi flags to bi.yml instead of call option * Fix after rebase * Fix after rebase * Fix after rebase * Fix after rebase * Airbyte integration (#86) * [DATA-366] Update datahub package and add dbt extra (#79) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Added IAP Airbyte authorization with service account file Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> * Service account OIDC ID token in deploy command (#91) * Added possibility to pass platform-agnostic service account OIDC ID token to deploy command * Added changelog information * added 'source freshness' command to dbt compile * Bump copier (#93) * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * dp compile default environment has been set to local * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump datahub (#84) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump packages except copier * fix: package name * chore: add specific version for pyyaml * fix: misspelling * chore: test docs * chore: test * test * test * test * test * test * test * chore: bump packages * chore: add renovate * chore: bump copier * chore: update exception * fix: update by removing not existing flag * chore: update tests * feat: update schema on manifest to v7 * chore: bump packages * chore: delete renovate * chore: bump python version * chore: remove junk * chore: bring back old exception * chore: add matrix to tox * chore: add new require and classifier * trigger * chore(docs): update badge and changelog * chore(docs): replace old extension * chore: add description * chore(docs): add old suffix and envops * chore(docs): update changelog * chore(docs): update changelog * chore(docs): update copier * chore: pinup version for dbt2looker * chore(docs): update changelog * chore: replace var for more meaningful Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> * chore: downgrade version to properly prepare release * chore(docs): fix date in changelog * BI deployment is disabled when config is missing (#95) * DATA-629: Added Airbyte configuration documentation (#97) * feat: disable sync bucket (#102) * chore: add flag to turn off bucket syncing with artefacts * chore: replace default value * chore(docs): updat changelog * fix: add readthedocs config file * chore: repair failing tests * feat: add test for sync_bucket * chore(docs): cosmetic changes in changelog * chore: rename `sync-bucket` to `disable-bucket-sync` * chore(docs): update with flag * chore: replace `sync_bucket` with `disable_bucket_sync` * chore: update name for `_bucket_sync` --------- Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * FIX #103 - Bump version and CHANGELOG for release 0.25.0 --------- Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Piotr Sierkin <psierkin@gmail.com> Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com>
Paweł Jurkiewicz (paszynski)
added a commit
that referenced
this pull request
May 16, 2023
* chore: downgrade version to properly prepare release * chore(docs): fix date in changelog * BI deployment is disabled when config is missing (#95) * DATA-629: Added Airbyte configuration documentation (#97) * feat: disable sync bucket (#102) * chore: add flag to turn off bucket syncing with artefacts * chore: replace default value * chore(docs): updat changelog * fix: add readthedocs config file * chore: repair failing tests * feat: add test for sync_bucket * chore(docs): cosmetic changes in changelog * chore: rename `sync-bucket` to `disable-bucket-sync` * chore(docs): update with flag * chore: replace `sync_bucket` with `disable_bucket_sync` * chore: update name for `_bucket_sync` --------- Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Fix failing readthedocs builds (#104) * Removed sphinx action from prepare-release pipeline (#105) * Bump werkzeug from 2.2.2 to 2.2.3 (#106) Bumps [werkzeug](https://github.com/pallets/werkzeug) from 2.2.2 to 2.2.3. - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@2.2.2...2.2.3) --- updated-dependencies: - dependency-name: werkzeug dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * bump codeclimate-action to 4.0.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> * FIX #108 - Bump version and CHANGELOG for release 0.25.1 * removed trailing characters from CHANGELOG.md --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Piotr Sierkin <psierkin@gmail.com> Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Jakub Szafran (jszafran)
added a commit
that referenced
this pull request
Jul 4, 2023
* Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * feat: looker integration for compile and deploy, new command: bi * tests fix, looker git with ssh * chore: organizing imports, removed not used variables * chore: dbt2looker as extra dependency, mypy errors fix, bi-git-key-path option instead of key-path * chore: formatting, types-requests dependency for mypy * refactor: refactoring bi, removing bi command, fix tests, moving bi flags to bi.yml instead of call option * Fix after rebase * Fix after rebase * Fix after rebase * Fix after rebase * Airbyte integration (#86) * [DATA-366] Update datahub package and add dbt extra (#79) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Added IAP Airbyte authorization with service account file Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> * Service account OIDC ID token in deploy command (#91) * Added possibility to pass platform-agnostic service account OIDC ID token to deploy command * Added changelog information * added 'source freshness' command to dbt compile * Bump copier (#93) * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * dp compile default environment has been set to local * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump datahub (#84) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump packages except copier * fix: package name * chore: add specific version for pyyaml * fix: misspelling * chore: test docs * chore: test * test * test * test * test * test * test * chore: bump packages * chore: add renovate * chore: bump copier * chore: update exception * fix: update by removing not existing flag * chore: update tests * feat: update schema on manifest to v7 * chore: bump packages * chore: delete renovate * chore: bump python version * chore: remove junk * chore: bring back old exception * chore: add matrix to tox * chore: add new require and classifier * trigger * chore(docs): update badge and changelog * chore(docs): replace old extension * chore: add description * chore(docs): add old suffix and envops * chore(docs): update changelog * chore(docs): update changelog * chore(docs): update copier * chore: pinup version for dbt2looker * chore(docs): update changelog * chore: replace var for more meaningful Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> * chore: downgrade version to properly prepare release * chore(docs): fix date in changelog * BI deployment is disabled when config is missing (#95) * DATA-629: Added Airbyte configuration documentation (#97) * feat: disable sync bucket (#102) * chore: add flag to turn off bucket syncing with artefacts * chore: replace default value * chore(docs): updat changelog * fix: add readthedocs config file * chore: repair failing tests * feat: add test for sync_bucket * chore(docs): cosmetic changes in changelog * chore: rename `sync-bucket` to `disable-bucket-sync` * chore(docs): update with flag * chore: replace `sync_bucket` with `disable_bucket_sync` * chore: update name for `_bucket_sync` --------- Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Fix failing readthedocs builds (#104) * Removed sphinx action from prepare-release pipeline (#105) * Bump werkzeug from 2.2.2 to 2.2.3 (#106) * Bump werkzeug from 2.2.2 to 2.2.3 Bumps [werkzeug](https://github.com/pallets/werkzeug) from 2.2.2 to 2.2.3. - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@2.2.2...2.2.3) --- updated-dependencies: - dependency-name: werkzeug dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * bump codeclimate-action to 4.0.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> * Airbyte fix creating multiple connections (#109) * Fixing tests. * Fix failing test. * Remove redundant line. * Add info about fix to CHANGELOG.md. * Add workspace_id entry to config. * Get rid of walrus operator & add test for missing workspace id. * Fix wrong connection id in Airflow ingestion task. * Fix typo in test name. * Add fetching default workspace if not available in config. * Increasign test coverage. * Update workspace_id configuration description. * Bumping Python version to 3.8 in prepare-release workflow. * Pin pydantic version (#112) * Pin Pydantic version until Copier fixes their dependency. * Add info to changelog. * FIX #113 - Bump version and CHANGELOG for release 0.25.3 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Piotr Sierkin <psierkin@gmail.com> Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jakub Szafran <33091690+jszafran@users.noreply.github.com> Co-authored-by: jszafran <jszafran.pv@gmail.com>
Michał Rudko (michalrudko)
added a commit
that referenced
this pull request
Aug 25, 2023
* Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * feat: looker integration for compile and deploy, new command: bi * tests fix, looker git with ssh * chore: organizing imports, removed not used variables * chore: dbt2looker as extra dependency, mypy errors fix, bi-git-key-path option instead of key-path * chore: formatting, types-requests dependency for mypy * refactor: refactoring bi, removing bi command, fix tests, moving bi flags to bi.yml instead of call option * Fix after rebase * Fix after rebase * Fix after rebase * Fix after rebase * Airbyte integration (#86) * [DATA-366] Update datahub package and add dbt extra (#79) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Added IAP Airbyte authorization with service account file Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> * Service account OIDC ID token in deploy command (#91) * Added possibility to pass platform-agnostic service account OIDC ID token to deploy command * Added changelog information * added 'source freshness' command to dbt compile * Bump copier (#93) * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * dp compile default environment has been set to local * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump datahub (#84) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump packages except copier * fix: package name * chore: add specific version for pyyaml * fix: misspelling * chore: test docs * chore: test * test * test * test * test * test * test * chore: bump packages * chore: add renovate * chore: bump copier * chore: update exception * fix: update by removing not existing flag * chore: update tests * feat: update schema on manifest to v7 * chore: bump packages * chore: delete renovate * chore: bump python version * chore: remove junk * chore: bring back old exception * chore: add matrix to tox * chore: add new require and classifier * trigger * chore(docs): update badge and changelog * chore(docs): replace old extension * chore: add description * chore(docs): add old suffix and envops * chore(docs): update changelog * chore(docs): update changelog * chore(docs): update copier * chore: pinup version for dbt2looker * chore(docs): update changelog * chore: replace var for more meaningful Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> * chore: downgrade version to properly prepare release * chore(docs): fix date in changelog * BI deployment is disabled when config is missing (#95) * DATA-629: Added Airbyte configuration documentation (#97) * feat: disable sync bucket (#102) * chore: add flag to turn off bucket syncing with artefacts * chore: replace default value * chore(docs): updat changelog * fix: add readthedocs config file * chore: repair failing tests * feat: add test for sync_bucket * chore(docs): cosmetic changes in changelog * chore: rename `sync-bucket` to `disable-bucket-sync` * chore(docs): update with flag * chore: replace `sync_bucket` with `disable_bucket_sync` * chore: update name for `_bucket_sync` --------- Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Fix failing readthedocs builds (#104) * Removed sphinx action from prepare-release pipeline (#105) * Bump werkzeug from 2.2.2 to 2.2.3 (#106) * Bump werkzeug from 2.2.2 to 2.2.3 Bumps [werkzeug](https://github.com/pallets/werkzeug) from 2.2.2 to 2.2.3. - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@2.2.2...2.2.3) --- updated-dependencies: - dependency-name: werkzeug dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * bump codeclimate-action to 4.0.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> * Airbyte fix creating multiple connections (#109) * Fixing tests. * Fix failing test. * Remove redundant line. * Add info about fix to CHANGELOG.md. * Add workspace_id entry to config. * Get rid of walrus operator & add test for missing workspace id. * Fix wrong connection id in Airflow ingestion task. * Fix typo in test name. * Add fetching default workspace if not available in config. * Increasign test coverage. * Update workspace_id configuration description. * Bumping Python version to 3.8 in prepare-release workflow. * Pin pydantic version (#112) * Pin Pydantic version until Copier fixes their dependency. * Add info to changelog. * Upgrade dbt to 1.5.4 (#114) * Upgrade dbt to 1.5.4 * Fixing imports for publish.py * Update of dbt_schema_version * Updating manifest.json for tests * Fix manifest for testing * Fixing manifest cntd. * Fixing manifest cntd. * Replace example-project with exampleproject * Adding tutorial for upgrading manifest in tests cntd. * Fixing typos in the docs * Version fixes before release --------- Co-authored-by: Michal Rudko <michal.rudko@getindata.com> * FIX #115 - Bump version and CHANGELOG for release 0.26.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Piotr Sierkin <psierkin@gmail.com> Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jakub Szafran <33091690+jszafran@users.noreply.github.com> Co-authored-by: jszafran <jszafran.pv@gmail.com> Co-authored-by: michalrudko <michal.rudko@gmail.com> Co-authored-by: Michal Rudko <michal.rudko@getindata.com>
Piotr Pękala (p-pekala)
added a commit
that referenced
this pull request
Dec 7, 2023
* Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * feat: looker integration for compile and deploy, new command: bi * tests fix, looker git with ssh * chore: organizing imports, removed not used variables * chore: dbt2looker as extra dependency, mypy errors fix, bi-git-key-path option instead of key-path * chore: formatting, types-requests dependency for mypy * refactor: refactoring bi, removing bi command, fix tests, moving bi flags to bi.yml instead of call option * Fix after rebase * Fix after rebase * Fix after rebase * Fix after rebase * Airbyte integration (#86) * [DATA-366] Update datahub package and add dbt extra (#79) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Added IAP Airbyte authorization with service account file Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> * Service account OIDC ID token in deploy command (#91) * Added possibility to pass platform-agnostic service account OIDC ID token to deploy command * Added changelog information * added 'source freshness' command to dbt compile * Bump copier (#93) * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * dp compile default environment has been set to local * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump datahub (#84) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump packages except copier * fix: package name * chore: add specific version for pyyaml * fix: misspelling * chore: test docs * chore: test * test * test * test * test * test * test * chore: bump packages * chore: add renovate * chore: bump copier * chore: update exception * fix: update by removing not existing flag * chore: update tests * feat: update schema on manifest to v7 * chore: bump packages * chore: delete renovate * chore: bump python version * chore: remove junk * chore: bring back old exception * chore: add matrix to tox * chore: add new require and classifier * trigger * chore(docs): update badge and changelog * chore(docs): replace old extension * chore: add description * chore(docs): add old suffix and envops * chore(docs): update changelog * chore(docs): update changelog * chore(docs): update copier * chore: pinup version for dbt2looker * chore(docs): update changelog * chore: replace var for more meaningful Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> * chore: downgrade version to properly prepare release * chore(docs): fix date in changelog * BI deployment is disabled when config is missing (#95) * DATA-629: Added Airbyte configuration documentation (#97) * feat: disable sync bucket (#102) * chore: add flag to turn off bucket syncing with artefacts * chore: replace default value * chore(docs): updat changelog * fix: add readthedocs config file * chore: repair failing tests * feat: add test for sync_bucket * chore(docs): cosmetic changes in changelog * chore: rename `sync-bucket` to `disable-bucket-sync` * chore(docs): update with flag * chore: replace `sync_bucket` with `disable_bucket_sync` * chore: update name for `_bucket_sync` --------- Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Fix failing readthedocs builds (#104) * Removed sphinx action from prepare-release pipeline (#105) * Bump werkzeug from 2.2.2 to 2.2.3 (#106) * Bump werkzeug from 2.2.2 to 2.2.3 Bumps [werkzeug](https://github.com/pallets/werkzeug) from 2.2.2 to 2.2.3. - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@2.2.2...2.2.3) --- updated-dependencies: - dependency-name: werkzeug dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * bump codeclimate-action to 4.0.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> * Airbyte fix creating multiple connections (#109) * Fixing tests. * Fix failing test. * Remove redundant line. * Add info about fix to CHANGELOG.md. * Add workspace_id entry to config. * Get rid of walrus operator & add test for missing workspace id. * Fix wrong connection id in Airflow ingestion task. * Fix typo in test name. * Add fetching default workspace if not available in config. * Increasign test coverage. * Update workspace_id configuration description. * Bumping Python version to 3.8 in prepare-release workflow. * Pin pydantic version (#112) * Pin Pydantic version until Copier fixes their dependency. * Add info to changelog. * Upgrade dbt to 1.5.4 (#114) * Upgrade dbt to 1.5.4 * Fixing imports for publish.py * Update of dbt_schema_version * Updating manifest.json for tests * Fix manifest for testing * Fixing manifest cntd. * Fixing manifest cntd. * Replace example-project with exampleproject * Adding tutorial for upgrading manifest in tests cntd. * Fixing typos in the docs * Version fixes before release --------- Co-authored-by: Michal Rudko <michal.rudko@getindata.com> * Databricks generate job (#116) * new feature * small fixes * dbt databricks factory requirement * python 3.9 onwards * testing * linting * annotations fix * linting * no docs * linting * update docs * testing * tests with databricks * testenv * Upgrade pyyaml * Dbt glue (#118) * DBT glue * DBT glue * DBT glue * DBT glue * FIX #119 - Bump version and CHANGELOG for release 0.27.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Piotr Sierkin <psierkin@gmail.com> Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jakub Szafran <33091690+jszafran@users.noreply.github.com> Co-authored-by: jszafran <jszafran.pv@gmail.com> Co-authored-by: michalrudko <michal.rudko@gmail.com> Co-authored-by: Michal Rudko <michal.rudko@getindata.com> Co-authored-by: ptutak-getindata <133227844+ptutak-getindata@users.noreply.github.com>
Piotr Pękala (p-pekala)
added a commit
that referenced
this pull request
Dec 8, 2023
* Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * feat: looker integration for compile and deploy, new command: bi * tests fix, looker git with ssh * chore: organizing imports, removed not used variables * chore: dbt2looker as extra dependency, mypy errors fix, bi-git-key-path option instead of key-path * chore: formatting, types-requests dependency for mypy * refactor: refactoring bi, removing bi command, fix tests, moving bi flags to bi.yml instead of call option * Fix after rebase * Fix after rebase * Fix after rebase * Fix after rebase * Airbyte integration (#86) * [DATA-366] Update datahub package and add dbt extra (#79) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Added IAP Airbyte authorization with service account file Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> * Service account OIDC ID token in deploy command (#91) * Added possibility to pass platform-agnostic service account OIDC ID token to deploy command * Added changelog information * added 'source freshness' command to dbt compile * Bump copier (#93) * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * dp compile default environment has been set to local * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump datahub (#84) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump packages except copier * fix: package name * chore: add specific version for pyyaml * fix: misspelling * chore: test docs * chore: test * test * test * test * test * test * test * chore: bump packages * chore: add renovate * chore: bump copier * chore: update exception * fix: update by removing not existing flag * chore: update tests * feat: update schema on manifest to v7 * chore: bump packages * chore: delete renovate * chore: bump python version * chore: remove junk * chore: bring back old exception * chore: add matrix to tox * chore: add new require and classifier * trigger * chore(docs): update badge and changelog * chore(docs): replace old extension * chore: add description * chore(docs): add old suffix and envops * chore(docs): update changelog * chore(docs): update changelog * chore(docs): update copier * chore: pinup version for dbt2looker * chore(docs): update changelog * chore: replace var for more meaningful Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> * chore: downgrade version to properly prepare release * chore(docs): fix date in changelog * BI deployment is disabled when config is missing (#95) * DATA-629: Added Airbyte configuration documentation (#97) * feat: disable sync bucket (#102) * chore: add flag to turn off bucket syncing with artefacts * chore: replace default value * chore(docs): updat changelog * fix: add readthedocs config file * chore: repair failing tests * feat: add test for sync_bucket * chore(docs): cosmetic changes in changelog * chore: rename `sync-bucket` to `disable-bucket-sync` * chore(docs): update with flag * chore: replace `sync_bucket` with `disable_bucket_sync` * chore: update name for `_bucket_sync` --------- Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Fix failing readthedocs builds (#104) * Removed sphinx action from prepare-release pipeline (#105) * Bump werkzeug from 2.2.2 to 2.2.3 (#106) * Bump werkzeug from 2.2.2 to 2.2.3 Bumps [werkzeug](https://github.com/pallets/werkzeug) from 2.2.2 to 2.2.3. - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@2.2.2...2.2.3) --- updated-dependencies: - dependency-name: werkzeug dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * bump codeclimate-action to 4.0.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> * Airbyte fix creating multiple connections (#109) * Fixing tests. * Fix failing test. * Remove redundant line. * Add info about fix to CHANGELOG.md. * Add workspace_id entry to config. * Get rid of walrus operator & add test for missing workspace id. * Fix wrong connection id in Airflow ingestion task. * Fix typo in test name. * Add fetching default workspace if not available in config. * Increasign test coverage. * Update workspace_id configuration description. * Bumping Python version to 3.8 in prepare-release workflow. * Pin pydantic version (#112) * Pin Pydantic version until Copier fixes their dependency. * Add info to changelog. * Upgrade dbt to 1.5.4 (#114) * Upgrade dbt to 1.5.4 * Fixing imports for publish.py * Update of dbt_schema_version * Updating manifest.json for tests * Fix manifest for testing * Fixing manifest cntd. * Fixing manifest cntd. * Replace example-project with exampleproject * Adding tutorial for upgrading manifest in tests cntd. * Fixing typos in the docs * Version fixes before release --------- Co-authored-by: Michal Rudko <michal.rudko@getindata.com> * Databricks generate job (#116) * new feature * small fixes * dbt databricks factory requirement * python 3.9 onwards * testing * linting * annotations fix * linting * no docs * linting * update docs * testing * tests with databricks * testenv * Upgrade pyyaml * Dbt glue (#118) * DBT glue * DBT glue * DBT glue * DBT glue * Dbt glue with spark (#120) * Dbt glue with spark * Fixing dp cli * FIX #121 - Bump version and CHANGELOG for release 0.28.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Piotr Sierkin <psierkin@gmail.com> Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jakub Szafran <33091690+jszafran@users.noreply.github.com> Co-authored-by: jszafran <jszafran.pv@gmail.com> Co-authored-by: michalrudko <michal.rudko@gmail.com> Co-authored-by: Michal Rudko <michal.rudko@getindata.com> Co-authored-by: ptutak-getindata <133227844+ptutak-getindata@users.noreply.github.com>
Piotr Pękala (p-pekala)
added a commit
that referenced
this pull request
Dec 8, 2023
* Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * feat: looker integration for compile and deploy, new command: bi * tests fix, looker git with ssh * chore: organizing imports, removed not used variables * chore: dbt2looker as extra dependency, mypy errors fix, bi-git-key-path option instead of key-path * chore: formatting, types-requests dependency for mypy * refactor: refactoring bi, removing bi command, fix tests, moving bi flags to bi.yml instead of call option * Fix after rebase * Fix after rebase * Fix after rebase * Fix after rebase * Airbyte integration (#86) * [DATA-366] Update datahub package and add dbt extra (#79) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Added IAP Airbyte authorization with service account file Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> * Service account OIDC ID token in deploy command (#91) * Added possibility to pass platform-agnostic service account OIDC ID token to deploy command * Added changelog information * added 'source freshness' command to dbt compile * Bump copier (#93) * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * dp compile default environment has been set to local * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump datahub (#84) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump packages except copier * fix: package name * chore: add specific version for pyyaml * fix: misspelling * chore: test docs * chore: test * test * test * test * test * test * test * chore: bump packages * chore: add renovate * chore: bump copier * chore: update exception * fix: update by removing not existing flag * chore: update tests * feat: update schema on manifest to v7 * chore: bump packages * chore: delete renovate * chore: bump python version * chore: remove junk * chore: bring back old exception * chore: add matrix to tox * chore: add new require and classifier * trigger * chore(docs): update badge and changelog * chore(docs): replace old extension * chore: add description * chore(docs): add old suffix and envops * chore(docs): update changelog * chore(docs): update changelog * chore(docs): update copier * chore: pinup version for dbt2looker * chore(docs): update changelog * chore: replace var for more meaningful Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> * chore: downgrade version to properly prepare release * chore(docs): fix date in changelog * BI deployment is disabled when config is missing (#95) * DATA-629: Added Airbyte configuration documentation (#97) * feat: disable sync bucket (#102) * chore: add flag to turn off bucket syncing with artefacts * chore: replace default value * chore(docs): updat changelog * fix: add readthedocs config file * chore: repair failing tests * feat: add test for sync_bucket * chore(docs): cosmetic changes in changelog * chore: rename `sync-bucket` to `disable-bucket-sync` * chore(docs): update with flag * chore: replace `sync_bucket` with `disable_bucket_sync` * chore: update name for `_bucket_sync` --------- Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Fix failing readthedocs builds (#104) * Removed sphinx action from prepare-release pipeline (#105) * Bump werkzeug from 2.2.2 to 2.2.3 (#106) * Bump werkzeug from 2.2.2 to 2.2.3 Bumps [werkzeug](https://github.com/pallets/werkzeug) from 2.2.2 to 2.2.3. - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@2.2.2...2.2.3) --- updated-dependencies: - dependency-name: werkzeug dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * bump codeclimate-action to 4.0.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> * Airbyte fix creating multiple connections (#109) * Fixing tests. * Fix failing test. * Remove redundant line. * Add info about fix to CHANGELOG.md. * Add workspace_id entry to config. * Get rid of walrus operator & add test for missing workspace id. * Fix wrong connection id in Airflow ingestion task. * Fix typo in test name. * Add fetching default workspace if not available in config. * Increasign test coverage. * Update workspace_id configuration description. * Bumping Python version to 3.8 in prepare-release workflow. * Pin pydantic version (#112) * Pin Pydantic version until Copier fixes their dependency. * Add info to changelog. * Upgrade dbt to 1.5.4 (#114) * Upgrade dbt to 1.5.4 * Fixing imports for publish.py * Update of dbt_schema_version * Updating manifest.json for tests * Fix manifest for testing * Fixing manifest cntd. * Fixing manifest cntd. * Replace example-project with exampleproject * Adding tutorial for upgrading manifest in tests cntd. * Fixing typos in the docs * Version fixes before release --------- Co-authored-by: Michal Rudko <michal.rudko@getindata.com> * Databricks generate job (#116) * new feature * small fixes * dbt databricks factory requirement * python 3.9 onwards * testing * linting * annotations fix * linting * no docs * linting * update docs * testing * tests with databricks * testenv * Upgrade pyyaml * Dbt glue (#118) * DBT glue * DBT glue * DBT glue * DBT glue * Dbt glue with spark (#120) * Dbt glue with spark * Fixing dp cli * Dbt upgtade (#122) * Dbt upgrade to 1.7 * Dbt upgrade to 1.7 * Fixing dp cli * FIX #123 - Bump version and CHANGELOG for release 0.29.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Piotr Sierkin <psierkin@gmail.com> Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jakub Szafran <33091690+jszafran@users.noreply.github.com> Co-authored-by: jszafran <jszafran.pv@gmail.com> Co-authored-by: michalrudko <michal.rudko@gmail.com> Co-authored-by: Michal Rudko <michal.rudko@getindata.com> Co-authored-by: ptutak-getindata <133227844+ptutak-getindata@users.noreply.github.com>
Piotr Pękala (p-pekala)
added a commit
that referenced
this pull request
Dec 8, 2023
* Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * feat: looker integration for compile and deploy, new command: bi * tests fix, looker git with ssh * chore: organizing imports, removed not used variables * chore: dbt2looker as extra dependency, mypy errors fix, bi-git-key-path option instead of key-path * chore: formatting, types-requests dependency for mypy * refactor: refactoring bi, removing bi command, fix tests, moving bi flags to bi.yml instead of call option * Fix after rebase * Fix after rebase * Fix after rebase * Fix after rebase * Airbyte integration (#86) * [DATA-366] Update datahub package and add dbt extra (#79) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Added IAP Airbyte authorization with service account file Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> * Service account OIDC ID token in deploy command (#91) * Added possibility to pass platform-agnostic service account OIDC ID token to deploy command * Added changelog information * added 'source freshness' command to dbt compile * Bump copier (#93) * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * dp compile default environment has been set to local * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump datahub (#84) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump packages except copier * fix: package name * chore: add specific version for pyyaml * fix: misspelling * chore: test docs * chore: test * test * test * test * test * test * test * chore: bump packages * chore: add renovate * chore: bump copier * chore: update exception * fix: update by removing not existing flag * chore: update tests * feat: update schema on manifest to v7 * chore: bump packages * chore: delete renovate * chore: bump python version * chore: remove junk * chore: bring back old exception * chore: add matrix to tox * chore: add new require and classifier * trigger * chore(docs): update badge and changelog * chore(docs): replace old extension * chore: add description * chore(docs): add old suffix and envops * chore(docs): update changelog * chore(docs): update changelog * chore(docs): update copier * chore: pinup version for dbt2looker * chore(docs): update changelog * chore: replace var for more meaningful Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> * chore: downgrade version to properly prepare release * chore(docs): fix date in changelog * BI deployment is disabled when config is missing (#95) * DATA-629: Added Airbyte configuration documentation (#97) * feat: disable sync bucket (#102) * chore: add flag to turn off bucket syncing with artefacts * chore: replace default value * chore(docs): updat changelog * fix: add readthedocs config file * chore: repair failing tests * feat: add test for sync_bucket * chore(docs): cosmetic changes in changelog * chore: rename `sync-bucket` to `disable-bucket-sync` * chore(docs): update with flag * chore: replace `sync_bucket` with `disable_bucket_sync` * chore: update name for `_bucket_sync` --------- Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Fix failing readthedocs builds (#104) * Removed sphinx action from prepare-release pipeline (#105) * Bump werkzeug from 2.2.2 to 2.2.3 (#106) * Bump werkzeug from 2.2.2 to 2.2.3 Bumps [werkzeug](https://github.com/pallets/werkzeug) from 2.2.2 to 2.2.3. - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@2.2.2...2.2.3) --- updated-dependencies: - dependency-name: werkzeug dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * bump codeclimate-action to 4.0.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> * Airbyte fix creating multiple connections (#109) * Fixing tests. * Fix failing test. * Remove redundant line. * Add info about fix to CHANGELOG.md. * Add workspace_id entry to config. * Get rid of walrus operator & add test for missing workspace id. * Fix wrong connection id in Airflow ingestion task. * Fix typo in test name. * Add fetching default workspace if not available in config. * Increasign test coverage. * Update workspace_id configuration description. * Bumping Python version to 3.8 in prepare-release workflow. * Pin pydantic version (#112) * Pin Pydantic version until Copier fixes their dependency. * Add info to changelog. * Upgrade dbt to 1.5.4 (#114) * Upgrade dbt to 1.5.4 * Fixing imports for publish.py * Update of dbt_schema_version * Updating manifest.json for tests * Fix manifest for testing * Fixing manifest cntd. * Fixing manifest cntd. * Replace example-project with exampleproject * Adding tutorial for upgrading manifest in tests cntd. * Fixing typos in the docs * Version fixes before release --------- Co-authored-by: Michal Rudko <michal.rudko@getindata.com> * Databricks generate job (#116) * new feature * small fixes * dbt databricks factory requirement * python 3.9 onwards * testing * linting * annotations fix * linting * no docs * linting * update docs * testing * tests with databricks * testenv * Upgrade pyyaml * Dbt glue (#118) * DBT glue * DBT glue * DBT glue * DBT glue * Dbt glue with spark (#120) * Dbt glue with spark * Fixing dp cli * Dbt upgtade (#122) * Dbt upgrade to 1.7 * Dbt upgrade to 1.7 * Fixing dp cli * Upgrade versions * Upgrade versions * FIX #124 - Bump version and CHANGELOG for release 0.30.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Piotr Sierkin <psierkin@gmail.com> Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jakub Szafran <33091690+jszafran@users.noreply.github.com> Co-authored-by: jszafran <jszafran.pv@gmail.com> Co-authored-by: michalrudko <michal.rudko@gmail.com> Co-authored-by: Michal Rudko <michal.rudko@getindata.com> Co-authored-by: ptutak-getindata <133227844+ptutak-getindata@users.noreply.github.com>
Dominik Gniewek-Węgrzyn (dgniewek)
added a commit
that referenced
this pull request
Nov 3, 2025
* Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * feat: looker integration for compile and deploy, new command: bi * tests fix, looker git with ssh * chore: organizing imports, removed not used variables * chore: dbt2looker as extra dependency, mypy errors fix, bi-git-key-path option instead of key-path * chore: formatting, types-requests dependency for mypy * refactor: refactoring bi, removing bi command, fix tests, moving bi flags to bi.yml instead of call option * Fix after rebase * Fix after rebase * Fix after rebase * Fix after rebase * Airbyte integration (#86) * [DATA-366] Update datahub package and add dbt extra (#79) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Added IAP Airbyte authorization with service account file Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> * Service account OIDC ID token in deploy command (#91) * Added possibility to pass platform-agnostic service account OIDC ID token to deploy command * Added changelog information * added 'source freshness' command to dbt compile * Bump copier (#93) * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * dp compile default environment has been set to local * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump datahub (#84) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump packages except copier * fix: package name * chore: add specific version for pyyaml * fix: misspelling * chore: test docs * chore: test * test * test * test * test * test * test * chore: bump packages * chore: add renovate * chore: bump copier * chore: update exception * fix: update by removing not existing flag * chore: update tests * feat: update schema on manifest to v7 * chore: bump packages * chore: delete renovate * chore: bump python version * chore: remove junk * chore: bring back old exception * chore: add matrix to tox * chore: add new require and classifier * trigger * chore(docs): update badge and changelog * chore(docs): replace old extension * chore: add description * chore(docs): add old suffix and envops * chore(docs): update changelog * chore(docs): update changelog * chore(docs): update copier * chore: pinup version for dbt2looker * chore(docs): update changelog * chore: replace var for more meaningful Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> * chore: downgrade version to properly prepare release * chore(docs): fix date in changelog * BI deployment is disabled when config is missing (#95) * DATA-629: Added Airbyte configuration documentation (#97) * feat: disable sync bucket (#102) * chore: add flag to turn off bucket syncing with artefacts * chore: replace default value * chore(docs): updat changelog * fix: add readthedocs config file * chore: repair failing tests * feat: add test for sync_bucket * chore(docs): cosmetic changes in changelog * chore: rename `sync-bucket` to `disable-bucket-sync` * chore(docs): update with flag * chore: replace `sync_bucket` with `disable_bucket_sync` * chore: update name for `_bucket_sync` --------- Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Fix failing readthedocs builds (#104) * Removed sphinx action from prepare-release pipeline (#105) * Bump werkzeug from 2.2.2 to 2.2.3 (#106) * Bump werkzeug from 2.2.2 to 2.2.3 Bumps [werkzeug](https://github.com/pallets/werkzeug) from 2.2.2 to 2.2.3. - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@2.2.2...2.2.3) --- updated-dependencies: - dependency-name: werkzeug dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * bump codeclimate-action to 4.0.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> * Airbyte fix creating multiple connections (#109) * Fixing tests. * Fix failing test. * Remove redundant line. * Add info about fix to CHANGELOG.md. * Add workspace_id entry to config. * Get rid of walrus operator & add test for missing workspace id. * Fix wrong connection id in Airflow ingestion task. * Fix typo in test name. * Add fetching default workspace if not available in config. * Increasign test coverage. * Update workspace_id configuration description. * Bumping Python version to 3.8 in prepare-release workflow. * Pin pydantic version (#112) * Pin Pydantic version until Copier fixes their dependency. * Add info to changelog. * Upgrade dbt to 1.5.4 (#114) * Upgrade dbt to 1.5.4 * Fixing imports for publish.py * Update of dbt_schema_version * Updating manifest.json for tests * Fix manifest for testing * Fixing manifest cntd. * Fixing manifest cntd. * Replace example-project with exampleproject * Adding tutorial for upgrading manifest in tests cntd. * Fixing typos in the docs * Version fixes before release --------- Co-authored-by: Michal Rudko <michal.rudko@getindata.com> * Databricks generate job (#116) * new feature * small fixes * dbt databricks factory requirement * python 3.9 onwards * testing * linting * annotations fix * linting * no docs * linting * update docs * testing * tests with databricks * testenv * Upgrade pyyaml * Dbt glue (#118) * DBT glue * DBT glue * DBT glue * DBT glue * Dbt glue with spark (#120) * Dbt glue with spark * Fixing dp cli * Dbt upgtade (#122) * Dbt upgrade to 1.7 * Dbt upgrade to 1.7 * Fixing dp cli * Upgrade versions * Upgrade versions * feat: Add support for python 3.11 and 3.12 (#125) # Add Python 3.11 and 3.12 Support ## Summary Adds full support for Python 3.11 and 3.12 while maintaining Python 3.9/3.10 compatibility. Python 3.9 is deprecated (EOL Oct 2025) with support ending April 30, 2026. ## Changes **Dependencies** - Removed `pydantic<2` (unused, blocked Python 3.12) - Updated `packaging` to `>=23.0` (setuptools compatibility) - Updated `fsspec`, `gcsfs`, `s3fs` to `>=2024.6.0,<2025.0.0` - Added `setuptools<75.0.0` constraint in tox/requirements-dev **Testing** - Configured tox for `py39`, `py310`, `py311`, `py312` - All 112 tests pass on Python 3.10-3.12 (96% coverage) - Updated pre-commit: flake8 6.1.0, mypy 1.7.1 **Code Quality** - Fixed exception handling: `ClientError` instead of bare `Exception` - Removed unnecessary `type: ignore` comments - Fixed line length violations **Documentation** - Updated `CONTRIBUTING.md` with dev setup and tox usage - Updated `.pre-commit-config.yaml` tool versions ## Testing ```bash tox # All versions tox -e py312 # Specific version Results: 112 tests passed, 96% coverage on Python 3.9-3.12 Release Version: 0.31.0 Type: Minor (non-breaking, adds Python 3.11/3.12 support) Target Branch: develop → main Breaking Changes None. Fully backward compatible. * fix: rollback manual version bump and changelog edit * fix: rollback another manual change in dp version * ci: rollback missing default CHANGELOG configuration that was manually edited * ci: fix release pipeline, add permissions to GH token * ci: fix release pipeline, add permissions to GH token * FIX #126 - Bump version and CHANGELOG for release 0.31.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Piotr Sierkin <psierkin@gmail.com> Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jakub Szafran <33091690+jszafran@users.noreply.github.com> Co-authored-by: jszafran <jszafran.pv@gmail.com> Co-authored-by: michalrudko <michal.rudko@gmail.com> Co-authored-by: Michal Rudko <michal.rudko@getindata.com> Co-authored-by: ptutak-getindata <133227844+ptutak-getindata@users.noreply.github.com> Co-authored-by: Dominik Gniewek-Węgrzyn <47598580+dgniewek@users.noreply.github.com> Co-authored-by: Dominik Gniewek-Węgrzyn <dominik.gniewek@getindata.com>
Dominik Gniewek-Węgrzyn (dgniewek)
added a commit
that referenced
this pull request
Nov 12, 2025
* Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * feat: looker integration for compile and deploy, new command: bi * tests fix, looker git with ssh * chore: organizing imports, removed not used variables * chore: dbt2looker as extra dependency, mypy errors fix, bi-git-key-path option instead of key-path * chore: formatting, types-requests dependency for mypy * refactor: refactoring bi, removing bi command, fix tests, moving bi flags to bi.yml instead of call option * Fix after rebase * Fix after rebase * Fix after rebase * Fix after rebase * Airbyte integration (#86) * [DATA-366] Update datahub package and add dbt extra (#79) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Added IAP Airbyte authorization with service account file Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> * Service account OIDC ID token in deploy command (#91) * Added possibility to pass platform-agnostic service account OIDC ID token to deploy command * Added changelog information * added 'source freshness' command to dbt compile * Bump copier (#93) * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * dp compile default environment has been set to local * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump datahub (#84) Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * [DATA-366] Update datahub package and add dbt extra (#79) * feat: update datahub package and add dbt extra * fix: replace datahub requirements * fix: pre-commit coma Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Documentation improvement (#80) * Documentation improvement * GitPython is not required anymore (#83) Installation docs upgrade * chore: bump packages except copier * fix: package name * chore: add specific version for pyyaml * fix: misspelling * chore: test docs * chore: test * test * test * test * test * test * test * chore: bump packages * chore: add renovate * chore: bump copier * chore: update exception * fix: update by removing not existing flag * chore: update tests * feat: update schema on manifest to v7 * chore: bump packages * chore: delete renovate * chore: bump python version * chore: remove junk * chore: bring back old exception * chore: add matrix to tox * chore: add new require and classifier * trigger * chore(docs): update badge and changelog * chore(docs): replace old extension * chore: add description * chore(docs): add old suffix and envops * chore(docs): update changelog * chore(docs): update changelog * chore(docs): update copier * chore: pinup version for dbt2looker * chore(docs): update changelog * chore: replace var for more meaningful Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> * chore: downgrade version to properly prepare release * chore(docs): fix date in changelog * BI deployment is disabled when config is missing (#95) * DATA-629: Added Airbyte configuration documentation (#97) * feat: disable sync bucket (#102) * chore: add flag to turn off bucket syncing with artefacts * chore: replace default value * chore(docs): updat changelog * fix: add readthedocs config file * chore: repair failing tests * feat: add test for sync_bucket * chore(docs): cosmetic changes in changelog * chore: rename `sync-bucket` to `disable-bucket-sync` * chore(docs): update with flag * chore: replace `sync_bucket` with `disable_bucket_sync` * chore: update name for `_bucket_sync` --------- Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> * Fix failing readthedocs builds (#104) * Removed sphinx action from prepare-release pipeline (#105) * Bump werkzeug from 2.2.2 to 2.2.3 (#106) * Bump werkzeug from 2.2.2 to 2.2.3 Bumps [werkzeug](https://github.com/pallets/werkzeug) from 2.2.2 to 2.2.3. - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@2.2.2...2.2.3) --- updated-dependencies: - dependency-name: werkzeug dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * bump codeclimate-action to 4.0.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> * Airbyte fix creating multiple connections (#109) * Fixing tests. * Fix failing test. * Remove redundant line. * Add info about fix to CHANGELOG.md. * Add workspace_id entry to config. * Get rid of walrus operator & add test for missing workspace id. * Fix wrong connection id in Airflow ingestion task. * Fix typo in test name. * Add fetching default workspace if not available in config. * Increasign test coverage. * Update workspace_id configuration description. * Bumping Python version to 3.8 in prepare-release workflow. * Pin pydantic version (#112) * Pin Pydantic version until Copier fixes their dependency. * Add info to changelog. * Upgrade dbt to 1.5.4 (#114) * Upgrade dbt to 1.5.4 * Fixing imports for publish.py * Update of dbt_schema_version * Updating manifest.json for tests * Fix manifest for testing * Fixing manifest cntd. * Fixing manifest cntd. * Replace example-project with exampleproject * Adding tutorial for upgrading manifest in tests cntd. * Fixing typos in the docs * Version fixes before release --------- Co-authored-by: Michal Rudko <michal.rudko@getindata.com> * Databricks generate job (#116) * new feature * small fixes * dbt databricks factory requirement * python 3.9 onwards * testing * linting * annotations fix * linting * no docs * linting * update docs * testing * tests with databricks * testenv * Upgrade pyyaml * Dbt glue (#118) * DBT glue * DBT glue * DBT glue * DBT glue * Dbt glue with spark (#120) * Dbt glue with spark * Fixing dp cli * Dbt upgtade (#122) * Dbt upgrade to 1.7 * Dbt upgrade to 1.7 * Fixing dp cli * Upgrade versions * Upgrade versions * feat: Add support for python 3.11 and 3.12 (#125) # Add Python 3.11 and 3.12 Support ## Summary Adds full support for Python 3.11 and 3.12 while maintaining Python 3.9/3.10 compatibility. Python 3.9 is deprecated (EOL Oct 2025) with support ending April 30, 2026. ## Changes **Dependencies** - Removed `pydantic<2` (unused, blocked Python 3.12) - Updated `packaging` to `>=23.0` (setuptools compatibility) - Updated `fsspec`, `gcsfs`, `s3fs` to `>=2024.6.0,<2025.0.0` - Added `setuptools<75.0.0` constraint in tox/requirements-dev **Testing** - Configured tox for `py39`, `py310`, `py311`, `py312` - All 112 tests pass on Python 3.10-3.12 (96% coverage) - Updated pre-commit: flake8 6.1.0, mypy 1.7.1 **Code Quality** - Fixed exception handling: `ClientError` instead of bare `Exception` - Removed unnecessary `type: ignore` comments - Fixed line length violations **Documentation** - Updated `CONTRIBUTING.md` with dev setup and tox usage - Updated `.pre-commit-config.yaml` tool versions ## Testing ```bash tox # All versions tox -e py312 # Specific version Results: 112 tests passed, 96% coverage on Python 3.9-3.12 Release Version: 0.31.0 Type: Minor (non-breaking, adds Python 3.11/3.12 support) Target Branch: develop → main Breaking Changes None. Fully backward compatible. * fix: rollback manual version bump and changelog edit * fix: rollback another manual change in dp version * ci: rollback missing default CHANGELOG configuration that was manually edited * ci: fix release pipeline, add permissions to GH token * ci: fix release pipeline, add permissions to GH token * ci: fix publish pipeline - pin a proper version * feat: Expand dbt version support (1.7.x - 1.10.x) and fix dbt 1.8+ compatibility (#127) * feat: support other dbt versions * chore: fix linter issue * FIX #128 - Bump version and CHANGELOG for release 0.32.0 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Piotr Pękala <piotr.pekala@getindata.com> Co-authored-by: Pawel Pinkos <pawel.pinkos@getindata.com> Co-authored-by: Piotr Sierkin <psierkin@gmail.com> Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com> Co-authored-by: Pawel Jurkiewicz <pawel.jurkiewicz94@gmail.com> Co-authored-by: pawelpinkos <113952441+pawelpinkos@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jakub Szafran <33091690+jszafran@users.noreply.github.com> Co-authored-by: jszafran <jszafran.pv@gmail.com> Co-authored-by: michalrudko <michal.rudko@gmail.com> Co-authored-by: Michal Rudko <michal.rudko@getindata.com> Co-authored-by: ptutak-getindata <133227844+ptutak-getindata@users.noreply.github.com> Co-authored-by: Dominik Gniewek-Węgrzyn <47598580+dgniewek@users.noreply.github.com> Co-authored-by: Dominik Gniewek-Węgrzyn <dominik.gniewek@getindata.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #101
changes:
--disable-bucket-syncflagKeep in mind: