Skip to content

Always run CI checks#17538

Merged
blaginin merged 6 commits intoapache:mainfrom
blaginin:db/always-runs-check
Sep 17, 2025
Merged

Always run CI checks#17538
blaginin merged 6 commits intoapache:mainfrom
blaginin:db/always-runs-check

Conversation

@blaginin
Copy link
Copy Markdown
Member

@blaginin blaginin commented Sep 12, 2025

Follow up on #17183

There are two ways to fix the "steps not run" problem:

  • 1: always run them. CI will become slower but we'll have merge queue anyway. On the positive side, we'll have consistent CI runs (for example, now when you just change *.md we won't run check configs.md and ***_functions.md is up-to-date although we should
  • 2: not run them (in a bit hacky way because of https://github.com/orgs/community/discussions/45899)

2 consumes less resources, so going with it for now

Example run: blaginin-org#3

Example md-only run: blaginin-org#4

@blaginin blaginin self-assigned this Sep 12, 2025
@github-actions github-actions Bot added the development-process Related to development process of DataFusion label Sep 12, 2025
@blaginin blaginin marked this pull request as ready for review September 14, 2025 16:48
@blaginin blaginin requested a review from alamb September 17, 2025 01:39
Comment on lines +832 to +855
needs: [ linux-rustdoc,
linux-wasm-pack,
linux-build-lib,
macos-aarch64,
vendor,
check-fmt,
msrv,
linux-datafusion-common-features,
linux-datafusion-substrait-features,
linux-datafusion-proto-features,
linux-cargo-check-datafusion,
linux-cargo-check-datafusion-functions,
linux-test,
linux-test-example,
linux-test-doc,
verify-benchmark-results,
sqllogictest-postgres,
sqllogictest-substrait,
test-datafusion-pyarrow,
clippy,
cargo-toml-formatting-checks,
linux-test-datafusion-cli,
config-docs-check
]
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should have a test (eg in rust, using a yaml parser) to validate this list is complete.

Copy link
Copy Markdown
Member Author

@blaginin blaginin Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed! will add on top

@blaginin blaginin added this pull request to the merge queue Sep 17, 2025
Merged via the queue into apache:main with commit 3c021ac Sep 17, 2025
5 checks passed
blaginin added a commit that referenced this pull request Sep 17, 2025
Humbedooh pushed a commit that referenced this pull request Sep 17, 2025
github-merge-queue Bot pushed a commit that referenced this pull request Apr 2, 2026
)

Related to #6880

Follow up on #17538

Bringing back the merge queues 🤞🏻
Last time it was painful because: 
- there was no `.asf.yaml` validation
- if something goes wrong, we weren't able to force merge without the
infra team

this PR solved the first problem. As for the second one, I have the
permissions to bypass so should be able to quickly revert if something
ever goes bad

<img width="830" height="151" alt="image"
src="https://github.com/user-attachments/assets/62fb2bb5-44d8-4aae-a006-188d96728140"
/>


Check result:
https://github.com/apache/datafusion/actions/runs/23715604583/job/69082077684?pr=21239

I also checked that CI will keep working by
[merging](apache/datafusion-sandbox#197) this
into our sandbox and then opening and merging [a dummy
pr](apache/datafusion-sandbox#204)

For now, bringing just the basic checks (in dev.yml). Will do rust.yml
separately if everything goes smoothly after this one is merged
zhuqi-lucas pushed a commit to zhuqi-lucas/arrow-datafusion that referenced this pull request Apr 4, 2026
…che#21239)

Related to apache#6880

Follow up on apache#17538

Bringing back the merge queues 🤞🏻
Last time it was painful because: 
- there was no `.asf.yaml` validation
- if something goes wrong, we weren't able to force merge without the
infra team

this PR solved the first problem. As for the second one, I have the
permissions to bypass so should be able to quickly revert if something
ever goes bad

<img width="830" height="151" alt="image"
src="https://github.com/user-attachments/assets/62fb2bb5-44d8-4aae-a006-188d96728140"
/>


Check result:
https://github.com/apache/datafusion/actions/runs/23715604583/job/69082077684?pr=21239

I also checked that CI will keep working by
[merging](apache/datafusion-sandbox#197) this
into our sandbox and then opening and merging [a dummy
pr](apache/datafusion-sandbox#204)

For now, bringing just the basic checks (in dev.yml). Will do rust.yml
separately if everything goes smoothly after this one is merged
Rich-T-kid pushed a commit to Rich-T-kid/datafusion that referenced this pull request Apr 21, 2026
…che#21239)

Related to apache#6880

Follow up on apache#17538

Bringing back the merge queues 🤞🏻
Last time it was painful because: 
- there was no `.asf.yaml` validation
- if something goes wrong, we weren't able to force merge without the
infra team

this PR solved the first problem. As for the second one, I have the
permissions to bypass so should be able to quickly revert if something
ever goes bad

<img width="830" height="151" alt="image"
src="https://github.com/user-attachments/assets/62fb2bb5-44d8-4aae-a006-188d96728140"
/>


Check result:
https://github.com/apache/datafusion/actions/runs/23715604583/job/69082077684?pr=21239

I also checked that CI will keep working by
[merging](apache/datafusion-sandbox#197) this
into our sandbox and then opening and merging [a dummy
pr](apache/datafusion-sandbox#204)

For now, bringing just the basic checks (in dev.yml). Will do rust.yml
separately if everything goes smoothly after this one is merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development-process Related to development process of DataFusion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants