Skip to content

DataFusion 49.0.0 release post #91

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alamb merged 50 commits into from
Jul 28, 2025
Merged

DataFusion 49.0.0 release post #91

Changes from 1 commit
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
0d5866f
DF 45 blog post
Omega359 Feb 20, 2025
7417e4c
Update content/blog/2025-02-20-datafusion-45.0.0.md
Omega359 Feb 21, 2025
3799609
Update content/blog/2025-02-20-datafusion-45.0.0.md
Omega359 Feb 21, 2025
88d7b6b
Set author to PMC.
Omega359 Feb 22, 2025
5a13332
Set author to PMC, incorporated feedback.
Omega359 Feb 22, 2025
b8de014
Update content/blog/2025-02-20-datafusion-45.0.0.md
Omega359 Feb 22, 2025
8a46200
expanded GSOC as it may not be obvious what it is and linked it up.
Omega359 Feb 22, 2025
5460e1a
Grammar fix.
Omega359 Feb 22, 2025
a2e3503
Typo fix
Omega359 Feb 22, 2025
e8e6734
Typo fix
Omega359 Feb 22, 2025
7bb8713
Adding spark functions to looking ahead section
Omega359 Feb 22, 2025
b330523
minor change
Omega359 Feb 22, 2025
3b9b11d
Fixed Jonah Gao's handle.
Omega359 Feb 24, 2025
29af566
Update content/blog/2025-02-20-datafusion-45.0.0.md
alamb Feb 25, 2025
d49a65c
WIP for DF 49 blog post.
Omega359 Jul 1, 2025
0167406
WIP for DF 49 blog post.
Omega359 Jul 1, 2025
6102ade
Update topK dynamic filtering perf section, cleanup the upgrade and c…
Omega359 Jul 1, 2025
cbbad27
Merge remote-tracking branch 'upstream/main' into origin_main
Omega359 Jul 6, 2025
3ece66e
DF 47.0.0 blog post
Omega359 Jul 6, 2025
ef46a35
Remove incomplete and accidentally added DF 49 blog post
Omega359 Jul 6, 2025
9e0f4e1
Fix header.
Omega359 Jul 6, 2025
4f049f4
Grammar fix
Omega359 Jul 6, 2025
9780da0
Minor formatting
Omega359 Jul 6, 2025
cdf50f8
Adding disabling of re-validation of spill files to performance impro…
Omega359 Jul 6, 2025
1478e3d
Merge remote-tracking branch 'apache/main' into Omega359/main
alamb Jul 9, 2025
3859c80
Formatting and wordsmithing
alamb Jul 9, 2025
a149ac8
tweaks
alamb Jul 9, 2025
d433e7d
Update content/blog/2025-07-10-datafusion-47.0.0.md
alamb Jul 9, 2025
af1c645
Fixed link.
Omega359 Jul 9, 2025
88ad7c6
Add datafusion-tracing crate mention and logo, make text more concrete
alamb Jul 11, 2025
5f17467
Claude edits
alamb Jul 11, 2025
44aa48f
Update publishing date
alamb Jul 11, 2025
23a1512
Merge branch 'apache:main' into main
Omega359 Jul 12, 2025
7df9b65
Merge branch 'refs/heads/main' into origin_main
Omega359 Jul 18, 2025
3364f88
Skeleton of DF 49 blog post
Omega359 Jul 18, 2025
bfbcd1a
Fix frontmatter, remove breaking changes section, add link to new blog
alamb Jul 18, 2025
5ef1876
Write up dynamic filtering
alamb Jul 19, 2025
55d0e24
Add performance chart
alamb Jul 19, 2025
0165501
Reorder sections, add new diagram
alamb Jul 21, 2025
b3e72e5
Add note on async udfs
alamb Jul 21, 2025
4d08441
update async section
alamb Jul 21, 2025
b0bb055
Add note abotu WITHINK GROUP
alamb Jul 21, 2025
c6f7642
note about parquet encryption
alamb Jul 21, 2025
44e6e3b
Add spill to disk and regex_instr
alamb Jul 21, 2025
ebb60dc
add regexp_instr
alamb Jul 21, 2025
4b2a29b
Adjust date
alamb Jul 21, 2025
15919bc
Small updates and typo fixes.
Omega359 Jul 22, 2025
aaa202f
Wordsmith / OCD obeses
alamb Jul 23, 2025
be89378
Gemini AI wordsmith / spelling / style
alamb Jul 23, 2025
0ce255f
update performance
alamb Jul 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Typo fix
  • Loading branch information
@Omega359
Omega359 committed Feb 22, 2025
commit a2e350341c064ea44bcdf995e51253027d58d0aa
4 changes: 2 additions & 2 deletions content/blog/2025-02-20-datafusion-45.0.0.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ In addition, DataFusion has been appearing publicly more and more, both online a
DataFusion hit a milestone in its development by becoming [the fastest single node engine]
for querying Apache Parquet files in [clickbench] benchmark for the 43.0.0 release. A [lot
of work] went into making this happen! While other engines have subsequently gotten faster,
displacing DataFusion from the top spot, DataFusion still remains near the top and we [are planing
displacing DataFusion from the top spot, DataFusion still remains near the top and we [are planning
more improvements].

<img
Expand All @@ -166,7 +166,7 @@ files due to [upstream work in the parquet reader]. Kudos to [@XiangpengHong], [
[the fastest single node engine]: https://datafusion.apache.org/blog/2024/11/18/datafusion-fastest-single-node-parquet-clickbench/
[clickbench]: https://benchmark.clickhouse.com/
[lot of work]: https://github.com/apache/datafusion/issues/12821
[are planing more improvements]: https://github.com/apache/datafusion/issues/14586
[are planning more improvements]: https://github.com/apache/datafusion/issues/14586
[integrating]: https://github.com/apache/datafusion/issues/10918
[Arrow StringView]: https://docs.rs/arrow/latest/arrow/array/struct.GenericByteViewArray.html
[multiple variable length columns in the `GROUP BY` clause]: https://github.com/apache/datafusion/issues/9403
Expand Down