mwaa-local-runner
-
AWS open source newsletter #189
Feb 18, 2024 | 14 minute read
Edition #189 Welcome to issue #189 of the AWS open source newsletter, the newsletter where we search high and low to provide you with the best open source on AWS content. As always, this week we start with a round up of some freshly baked new projects for you to practice your four freedoms. This week we have projects that help you find your RDS instances, automate tasks from your online Chime calls, a very nice visual file browser for your Amazon S3 buckets, a tool to help you track and manage copying files from your S3 storage buckets, an active-active multi region cluster solution for Redis, and more!
-
Using Finch to run Apache Airflow using mwaa-local-runner
Feb 12, 2024 | 9 minute read
I show you how you can use the Finch to run Apache Airflow using the mwaa-local-runner tool, and how you can do this for your applications too As some of you may know, I have been creating content on Apache Airflow for a few years now. One of the open source projects that AWS has produced to make it easier for developers to get started with Apache Airflow, is mwaa-local-runner.
-
AWS open source newsletter #154
Apr 24, 2023 | 22 minute read
April 24th, 2023 - Instalment #154 Welcome Hello and welcome to the AWS open source newsletter, #154, the newsletter that just keeps on giving….in this case, keeps giving you brand new open source projects to practice your four freedoms on. We have another great selection of projects for you as always, starting off with “cfn-teleport” an essential cli tool for Cloudformation users, “aither” an interesting collaborative development tool using virtualised desktops on containers, “tabular-column-semantic-search” a tool to help you find similar types of data in your data lakes, “resource-lister” and “komiser” tools that help you manage your AWS resources, “resource-utilization” helps you track your AWS resource utilisation, “iot-network-traffic-control-and-load-testing-simulator” an interesting load and chaos testing example, and more!
- oss-newsletter
- aws open source
- Apache Oozie
- Apache Airflow
- Deep Java Library
- DJL
- mwaa-local-runner
- MWAA
- Trusted Language Extensions for PostgreSQL
- Supabase
- PostgreSQL
- Jupyter
- Grafana
- Opus
- Papermill
- Apache Spark
- HiveQL
- Amazon EKS
- Kubernetes
- Amazon EMR
- RStudio
- USBGuard
- Amazon Corretto
- AWS Amplify
- Apache Hive Metastore
- LoRaWAN
- gMSA
- Python
- OpenSearch
- AWS Copilot
- Marten
- Flutter
-
Getting mwaa-local-runner up on AWS Cloud9
Apr 17, 2023 | 5 minute read
Here is a quick recipe if you are looking to get mwaa-local-runner up and running on your Cloud9 developer setup. This might not be the most optimised way, so I am very happy to received suggestions on how to improve this. What I will cover here is how to deploy mwaa-local-runner onto a standard Cloud9 IDE, deployed in a default VPC. Updating my AWS Cloud9 environment The first thing I needed to do was to increase the size of my local disk as Cloud9 only provides 10gb of storage.
-
Exploring Shell Launch Scripts on Managed Workflows for Apache Airflow (MWAA) and mwaa-local-runner
Apr 17, 2023 | 9 minute read
Managed Workflows for Apache Airflow (MWAA) recently launched a new feature that a lot of folk had been asking for, which was the ability to add additional libraries, binaries, or environment variables when launching Airflow workers. If you missed the announcement, Amazon MWAA now supports Shell Launch Scripts, this new capability allows you to easily do this by creating a script and then configuring your MWAA environments to use that script during the start-up phase.
-
VSCode and Apache Airflow
Feb 20, 2023 | 8 minute read
VSCode and Apache Airflow In this short post, I wanted to highlight how you can use a VSCode plugin to work with a local running instance of Apache Airflow to improve the developer experience. This post was inspired by a tweet from Kaxil Naik who was asking about what features developers are looking for when using VSCode and Pycharm and Apache Airflow. In this post I will show you how you can configure mwaa-local-runner, an open source project that provides you with an easy way to get a local Apache Airflow environment up and running (that is configuration wide, aligned to the Amazon Managed Workflows for Apache Airflow service MWAA), together with some VSCode plugins.