Security
-
Reading and writing data across different AWS accounts with Amazon Managed Workflows for Apache Airflow v2.x
Sep 7, 2021 | 13 minute read
Reading and writing data across different AWS accounts in you Apache Airflow DAGs As regular readers will know, I sometimes lurk in the Apache Airflow slack channel to see what is going on. If you are new to Apache Airflow, or want to get a deeper understanding then I highly recommend spending some time here. The community is super welcoming and eager to help new participants. It was during a recent session I came across an interesting problem that one of the builders was having, which was how to access (read/write) data in an S3 bucket which was in a different account to the one hosting Amazon Managed Workflows for Apache Airflow (MWAA).
-
Working with permissions in Amazon Managed Workflows for Apache Airflow
Jan 27, 2021 | 10 minute read
Part of a series of posts to support an up-coming online event, the Innovate AI/ML on February 24th, from 9:00am GMT - you can sign up here Part 1 - Installation and configuration of Managed Workflows for Apache Airflow Part 2 - Working with Permissions <- this post Part 3 - Accessing Amazon Managed Workflows for Apache Airflow environments Part 4 - Interacting with Amazon Managed Workflows for Apache Airflow via the command line Part 5 - A simple CI/CD system for your development workflow Part 6 - Monitoring and logging Part 7 - Automating a simple AI/ML pipeline with Apache Airflow In this post I will be covering Part 2, how to ensure that you control access to Apache Airflow following best practices such as default no access/least privilege.