Markus Suhr<p>Just caught up with the recent Delta Lake webinar, </p><p>> Revolutionizing Delta Lake workflows on AWS Lambda with Polars, DuckDB, Daft & Rust</p><p>Some interesting hints there regarding lightweight processing of big-ish data. Easy to relate to any other framework instead of Lambda, e.g. <a href="https://gruene.social/tags/ApacheAirflow" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ApacheAirflow</span></a> tasks</p><p><a href="https://youtu.be/BR9oFD0QMAs" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="">youtu.be/BR9oFD0QMAs</span><span class="invisible"></span></a></p><p><a href="https://gruene.social/tags/dataengineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>dataengineering</span></a> <a href="https://gruene.social/tags/datascience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>datascience</span></a> <a href="https://gruene.social/tags/duckdb" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>duckdb</span></a> <a href="https://gruene.social/tags/daft" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>daft</span></a> <a href="https://gruene.social/tags/polars" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>polars</span></a> <a href="https://gruene.social/tags/pandas" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>pandas</span></a> <a href="https://gruene.social/tags/python" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>python</span></a> <a href="https://gruene.social/tags/spark" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>spark</span></a> <a href="https://gruene.social/tags/deltalake" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>deltalake</span></a> <a href="https://gruene.social/tags/databricks" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>databricks</span></a> <a href="https://gruene.social/tags/airflow" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>airflow</span></a> <a href="https://gruene.social/tags/bigdata" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>bigdata</span></a> <a href="https://gruene.social/tags/smalldata" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>smalldata</span></a></p>