doeks
-
Self managed Apache Airflow with Data on EKS
Mar 22, 2023 | 11 minute read
I have written in the past about how you can get started with Apache Airflow using the AWS managed service, Managed Workflows for Apache Airflow. But what if you want to self managed Apache Airflow? When I speak with developers, there are sometimes reasons why a managed service might not fit their needs. Some of the common things that come up include: whether you need the increase level of access, a greater level of control of the configuration of Apache Airflow have the need to have the very latest versions or features of Apache Airflow if you have the need to run workflows that use more resources that managed services provide (for example, need significant compute) Total Cost Ownership One thing to consider when assessing managed vs self managed is the cost of the managed service against the total costs of you having to do the same thing.