skip to content W3C SVG

AWS open source newsletter #213

Edition #213- August 2025

Welcome to issue #213 of the AWS open source newsletter, the newsletter where I try and provide you the best open source on AWS content. As always, this edition has more great new projects to check out. In this edition, we have a nice selection of projects that help you migrate your CDK projects, a number of graphical and text interfaces for a number of backend systems and data, a look at the Valkey client for Swift, and the usual sampling of. cool demos (which of course feature generative AI, as why wouldn’t you!)

The projects will keep you busy until next month for sure, but we also have plenty of reading material in this months newsletter. In this edition we have featured projects that include DocumentDB, Strands Agent, Kubernetes, ArgoCD, KEDA, Spring AI, Bref, OpenTofu, AWS CDK, Karpenter, Apache Airflow, dbt, PostgreSQL, MySQL, MariaDB, Apache Kafka, Apache Flink, Grafana, Amazon EMR, HBase, InfluxDB, Powertools for AWS Lambda, LangGraph, Valkey, Cedar, Mountpoint for Amazon S3, OpenZFS, AWS Parallel Computing Service, and AWS Neuron. That should keep you all very busy!

Check out the list of contributors at the end of the newsletter, and as always, get in touch if you want me to feature your projects in this open source newsletter.

Make sure you check out

If you are using the AWS Tools for Powershell, then check out Announcing the end-of-support for AWS Tools for PowerShell v4 for important info and timelines.

Latest open source projects

The great thing about open source projects is that you can review the source code. If you like the look of these projects, make sure you that take a look at the code, and if it is useful to you, get in touch with the maintainer to provide feedback, suggestions or even submit a contribution. The projects mentioned here do not represent any formal recommendation or endorsement, I am just sharing for greater awareness as I think they look useful and interesting!

Tools

TerraTitan

TerraTitan is an open source (GPL) LLM powered workflow that converts AWS CDK constructs to Terraform CDK (CDKTF). This tool is a major component used to generate and maintain the TerraConstructs library, providing reliable and trustworthy infrastructure components for everyone to use. There is also a workshop that will help you get started if this looks interesting to you.

aws-size

aws-size is a great tool that AWS Community Builder Jason Kao that checks hard to find size limits and usage for AWS which can help provide advance warning to teams about resource limits in AWS before it’s too late. Usage of these limits are not covered by AWS provided tooling such as Service Quotas and Trusted Advisor. Read the introduction blog post, Introducing aws-size: A New Open Source Tool for Difficult Service Limit Visibility in AWS to find out more.

iceberg_navigator

iceberg_navigator is a command line tool from AWS Community Builder Aki to makes it easy to navigate and inspect Apache Iceberg snapshot histories. It uses PyIceberg with AWS Glue REST Catalog to list, show details, and visualise snapshot lineage graphs for Iceberg tables stored on S3. He has also put together a blog post, Building a CLI Tool to Visualize AWS Iceberg Table Snapshot History that is a helpful walkthrough of the project and code.

valkey-swift

valkey-swift is a Swift client library for Valkey. README provides everything you Swifties need to know to get all that Valkey goodness in your apps.

q-view

q-view is a handy tool from AWS Community Builder Ryan Cormack that lets you visualise and analyse Amazon Q Developer conversation history from Q CLI. Check out the README for more details, and I will leave you with an example screenshot.

example screenshot of tool

graphc

graphc (stands for “graph console”) and is a really nice interactive text user interface from Dhruv Thakur that lets you query Neo4j/AWS Neptune databases via the command line. Check out the repo for screenshots and short animated videos of this in action.

csi-components

csi-components - This repository contains the tooling used to build minimal Amazon Linux based versions of the Kubernetes CSI Sidecars (and other related components such as the Kubernetes CSI snapshot-controller image). These images are used in the official releases of the EBS CSI Driver versions v1.45.0 and later.

Demos, Samples, Solutions and Workshops

QualityFlow

QualityFlow is a super interesting repo from Amazon Science that demonstrates the research paper, QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checks. Given the English description of a programming problem and a set of unit tests, the model’s goal is to synthesise the correct program that solves the problem and passes the tests. QualityFlow includes large language model (LLM) agents resembling a software development team, including code generation, testing, and self-debugging. We propose the LLM Quality Checker, which explicitly ``imagines'' whether the synthesised programs' execution would conform to the unit tests. The Quality Checks dynamically control the workflow, including actions to submit the final answer, clarify the problem statement, and revert previous workflow steps. Our experiments show that the Quality Checker can precisely accept any correct program, mitigate faulty synthesised tests, and prevent potential workflow deviation. QualityFlow establishes the state-of-the-art results on four program synthesis benchmarks: MBPP, HumanEval, and stricter evaluations from MBPP-EvalPlus and HumanEval-EvalPlus.

sample-cost-per-transaction

sample-cost-per-transaction - is a repo that provides a cost intelligence solution that correlates application performance traces with Infrastructure costs, providing unprecedented transaction-level visibility to drive strategic business decisions.

AWS services used in solution

sample-on-demand-workflow-orchestrator

sample-on-demand-workflow-orchestrator - this repo lets you build a serverless, dynamic workflow orchestration engine that is built with Amazon DynamoDB and AWS Lambda that enables flexible, event-driven task execution with complex dependency patterns.

solution architecture

strands-mcp-inter-agent

strands-mcp-inter-agent is some sample code from my colleague James Ward that shows you how MCP Agents can call other MCP Agents by exposing them as MCP Servers. his example uses a hierarchy of agents with where the outer agent calls (using MCP) an inner agent that does routing and prompt modification, which then calls another MCP server.

sample-sonic-java-playground

sample-sonic-java-playground - is a full-stack playground application for experimenting with Amazon’s NovaSonic API capabilities, built with React (v18.2.0) and Spring Boot (v3.2.0). The application supports invoking NovaSonic Speech to Speech model by configuring supported parameters.

sample-ai-agent-accelerator

sample-ai-agent-accelerator - if you wanted to get hands on with Amazon Bedrock AgentCore, then this repo is for you. This project is a sample reference implementation that showcases how to quickly build an AI agent using the Bedrock AgentCore building blocks. The implementation is fully serverless leveraging AgentCore Runtime, AgentCore Memory, AgentCore Observability, and Amazon S3 Vectors for Agentic RAG, eliminating the need to run databases.

The agent is built using the Strands Agent Python library and hosted on the AgentCore Runtime. It uses Strand’s built-in retrieve tool to perform semantic search using Bedrock Knowledge Bases, which ingests documents from an S3 bucket and stores the indexed vectors in S3 Vectors. User conversation state and history are fully managed by AgentCore Memory. Users interact with the agent via a web app that provides both a web GUI and an HTTP JSON API, hosted as a container on ECS Fargate behind an ALB. The web app is built using Python Flask and HTMX.

AWS and Community blog posts

This weeks essential reading

Here are the posts that I think are essential reads, so start here. We have more than we usually do, but thought these were all essential reads this month.

Community

Each month I spend time reading posts from across the AWS community on open source topics. In this section I share what personally caught my eye and interest, and I hope that many of you will also find them interesting.

This month we start off with AWS Community Builders Benjamen Pyle who takes a look at how to build cloud native event driven applications on Amazon EKS using KEDA, an open source event based auto-scaler, in his post KEDA to build Event-Driven Applications on EKS. Sample code is provided so you can follow along with that one. Sticking around in cloud native land we have AWS Community Builders Alejandro Velez who continues his series of posts that share production ready examples and best practices of how do automate your infrastructure configuration and deployment using GitOps with ArgoCD in the post, GitOps and IaC at Scale – ArgoCD and Open Tofu – Part 3 – Hardening and Manage users.

Regular readers of this newsletter will be familiar with the work of AWS Heroes Vadym Kazulkin, and he is back sharing more Java goodness in the first in a series of posts that look at Spring AI, Spring’s application framework for AI engineering, and how you can get this up and running with Amazon Bedrock. In the first post Spring AI with Amazon Bedrock - Part 1 Introduction and the sample application, he explores Spring AI and its concepts by building the conference tool application using Amazon Bedrock Converse API. Make sure you check the other posts in the series (part twowas published just before I put this together)

Many moons ago, I enjoyed developing PHP code and I am happy that the community continues to thrive (go 8.5 beta!). Bref is a PHP open source runtime for AWS Lambda, enabling serverless deployment of PHP applications. AWS Community Builder Paul Santus shows you in a very short post, Introducing Bref Lambda Layers Terraform module, how you can automate the deployment using Terraform. Keeping with the Terraform theme, although switching to open source, we have AWS Community Builder kvendingoldo who shares how set up ElastiCache Serverless with the Valkey engine by using OpenTofu in the post, How to create AWS Serverless Valkey via OpenTofu.

AWS Hero Kenta Goto looks at a topic that I know has affected me many times in my AWS CDK adventures, namely when you deploy without validating incorrect values, errors may occur during stack creation or updates. In How to Choose Validation Approaches in AWS CDK he dives into this topic and provides a number of suggestions on how to reduce these issues. Very nice post, I enjoyed this one. Sticking with AWS CDK we had the post Different ways to conditionally provision a CDK resource from Emmanuel Isenah that looks at another common use case that you will come up with as you develop your CDK stacks, namely how to deal with conditions when provisioning resources (in this case, Amazon S3 buckets) - I have had the exact same issue, so I definitely learned something reading this on how to improve my next CDK app.

Thats all for this month. If you have an open source article you want to share with the community, drop me a message (LinkedIn, or via ricsue@amazon.co.uk and I will be sure to check it out.

Cloud Native

example hybrid architecture

layer diagram for compute

Data and Analytics

example graph showing generation zgc

Other posts to check out

example app sequence diagram

Case Studies

Quick updates

Cedar

Amazon Verified Permissions now supports Cedar 4.5. This enables customers to use the latest Cedar features, including the “is” operator, which allows customers to grant access based on resource types. For example, in a petstore application, you can use the “is” operator to write a policy that only grants administrators permission to view a resource if that resource “is” an invoice. This addition enhances Cedar’s type system and helps catch potential type-related errors early in policy development. You can learn about other enhancements to Cedar on the Cedar releases page.

Apache Airflow

You can now downgrade to minor Apache Airflow versions on Amazon Managed Workflows for Apache Airflow (MWAA). Amazon MWAA is a managed orchestration service for Apache Airflow that makes it easier to set up and operate end-to-end data pipelines in the cloud. This in-place minor Apache Airflow version option allows you to downgrade your MWAA Apache Airflow version to any other supported minor version.

Apache Flink

Amazon Managed Service for Apache Flink simplifies the development and operation of real-time data stream processing applications by eliminating the complexity of managing Flink infrastructure. Apache Flink is an open source framework and engine for processing data streams. Amazon Managed Service for Apache Flink now supports Amazon Key Management Service (KMS) Customer Managed Keys (CMK). Amazon Managed Service for Apache Flink has always provided encryption by default using AWS-owned KMS keys. Now, customers have the option to use their own Customer Managed Keys providing greater control on how they can encrypt data stored in MSF.

PostgreSQL

A few updates for PostgreSQL fans.

First up is news that Amazon RDS for PostgreSQL now supports delayed read replicas, allowing you to specify a minimum time period that a replica database lags behind a source database. This feature creates a time buffer that helps protect against data loss from human errors such as accidental table drops or unintended data modifications. In disaster recovery scenarios, you can pause replication before problematic changes are applied, resume replication up to a specific log position, and promote the replica as your new primary database. This approach enables faster recovery compared to traditional point-in-time restore operations, which can take hours for large databases.

Following that is news that Amazon RDS for PostgreSQL 18 Beta 3 is now available in the Amazon RDS Database Preview Environment, allowing you to evaluate the pre-release of PostgreSQL 18 on Amazon RDS for PostgreSQL. You can deploy PostgreSQL 18 Beta 3 in the Amazon RDS Database Preview Environment that has the benefits of a fully managed database. PostgreSQL 18 includes “skip scan” support for multicolumn B-tree indexes and improves WHERE clause handling for OR and IN conditions. It introduces parallel GIN index builds and updates join operations. Observability improvements show buffer usage counts and index lookups during query execution, along with per-connection I/O utilisation metric.

MySQL

Amazon Aurora MySQL - Compatible Edition 3 (with MySQL 8.0 compatibility) now supports MySQL 8.0.42 through Aurora MySQL v3.10. In addition to several security enhancements and bug fixes, MySQL 8.0.42 contains performance improvements for parallel replication using writeset dependency tracking, as well as enhanced debugging capabilities within the InnoDB storage engine. Aurora MySQL 3.10 includes an increase in maximum storage capacity from 128 TiB to 256 TiB, allowing customers to manage larger database workloads within a single database cluster. Aurora MySQL 3.10 also introduces in-memory relay log optimisation that improves binary log replication performance by caching relay log content in memory, reducing commit latency and minimising storage I/O operations on binlog replicas. For more details, refer to the Aurora MySQL 3.10 and MySQL 8.0.42 release notes. To upgrade to Aurora MySQL 3.10, you can initiate a minor version upgrade manually by modifying your DB cluster, or you can enable the “Auto minor version upgrade” option when creating or modifying a DB cluster.

In addition to that, Amazon Relational Database Service (Amazon RDS) for MySQL now supports MySQL minor versions 8.0.43 and 8.4.6, the latest minors released by the MySQL community. We recommend upgrading to the newer minor versions to fix known security vulnerabilities in prior versions of MySQL and to benefit from bug fixes, performance improvements, and new functionality added by the MySQL community. Learn more about the enhancements in RDS for MySQL 8.0.43 and 8.4.6 in the Amazon RDS user guide. You can leverage automatic minor version upgrades to automatically upgrade your databases to more recent minor versions during scheduled maintenance windows. You can also use Amazon RDS Managed Blue/Green deployments for safer, simpler, and faster updates to your MySQL instances.

MariaDB

Some nice updates for MariaDB users.

Amazon Relational Database Service (Amazon RDS) for MariaDB now supports community MariaDB minor versions 11.4.8, 10.11.14 and 10.6.23. We recommend that you upgrade to the latest minor versions to fix known security vulnerabilities in prior versions of MariaDB, and to benefit from the bug fixes, performance improvements, and new functionality added by the MariaDB community. You can leverage automatic minor version upgrades to automatically upgrade your databases to more recent minor versions during scheduled maintenance windows. You can also leverage Amazon RDS Managed Blue/Green deployments for safer, simpler, and faster updates to your MariaDB instances.

In more news, Amazon RDS for MariaDB now supports MariaDB major version 11.8, the latest long-term maintenance release from the MariaDB community. This release supports MariaDB 11.8.3 minor version. Amazon RDS for MariaDB 11.8 now supports the MariaDB Vector feature, allowing you to store vector embeddings in your database and use retrieval-augmented generation (RAG) when building your Artificial Intelligence (AI) applications. You can use MariaDB Vector to build generative AI capabilities into your e-commerce, media, health applications, and more to find similar items within a catalog. MariaDB 11.8 also introduces the ability to limit maximum size of temporary files and tables, allowing you to better manage your databases’ storage and prevent potential issues caused by oversized temporary objects. You can initiate a major version upgrade by manually modifying your DB cluster, by restoring a snapshot or by leveraging Amazon RDS Managed Blue/Green deployments to upgrade your databases to RDS for MariaDB 11.8.

Mountpoint for Amazon S3

Mountpoint for Amazon S3 Container Storage Interface (CSI) driver now accelerates performance for repeatedly accessed data, adds support for Security-Enhanced Linux (SELinux) mount options, and simplifies logging and permissions management. The latest version of the Mountpoint for Amazon S3 CSI driver (v2) introduces four key capabilities. First, it adds support for caching data across multiple pods. By using the new caching capabilities in Mountpoint for Amazon S3 CSI driver v2, you can finish large-scale financial simulation jobs up to 2x faster by eliminating the overhead of multiple pods individually caching the same data. Second, you can now run your Kubernetes applications on SELinux-enabled environments like Red Hat OpenShift. Third, it lets you use Amazon EKS Pod Identity to simplify how you manage access policies across Amazon EKS clusters, including cross-account access. Fourth, it simplifies how you access logs and get insights into your mounts by using kubectl, a command line tool.

OpenZFS

Amazon FSx now offers customers the option to use Internet Protocol version 6 (IPv6) for access to Amazon FSx for OpenZFS file systems. More and more customers are adopting IPv6 to mitigate IPv4 address exhaustion in their private networks or to satisfy government mandates such as the US Office of Management and Budget (OMB) M-21-07 memorandum. With this launch, customers can now access their file systems using IPv4, IPv6, or dual-stack clients without the need for complex infrastructure to handle IPv6 to IPv4 address translation.

AWS Parallel Computing Service

AWS Parallel Computing Service (AWS PCS) now supports SPANK (Slurm Plug-in Architecture for Node and job [K]control) plugins, enabling you to extend and modify how Slurm schedules and processes your high performance computing (HPC) workloads without modifying Slurm directly. Using SPANK plugins, you can now integrate AWS PCS with container technologies, implement custom monitoring of memory and I/O patterns, and dynamically modify job launches to enhance resource management. For example, you can use Enroot and Pyxis plugins to seamlessly run containerised machine learning and HPC workloads using images from Amazon Elastic Container Registry, Docker Hub, NVIDIA NGC, or other container registries. (as a side note, if you are sniggering reading this, you are not alone).

AWS Neuron

AWS announced the general availability of Neuron SDK 2.25.0, delivering improvements for inference workloads and performance monitoring on AWS Inferentia and Trainium instances. This latest release adds context and data parallelism support as well as chunked attention for long sequence processing in inference, and updates the neuron-ls and neuron-monitor APIs with more information on node affinities and device utilisation, respectively. This release also introduces automatic aliasing (Beta) for fast tensor operations, and adds improvements for disaggregated serving (Beta). Finally, it provides upgraded AMIs and Deep Learning Containers for inference and training workloads on Neuron.

Kubernetes

A few updates for cloud native folk running Kubernetes.

Amazon Elastic Kubernetes Service (Amazon EKS) now supports Kubernetes namespace configuration for AWS and Community add-ons, providing you greater control over how add-ons are organised within your Kubernetes cluster. With namespace configuration, you can now specify a custom namespace during add-on installation, enabling better organiastion and isolation of add-on objects within your EKS cluster. This flexibility helps you align add-ons with your operational needs and existing namespace strategy. Once an add-on is installed in a specific namespace, you must remove and recreate the add-on to change its namespace. This feature is available through the AWS Management Console, Amazon EKS APIs, AWS Command Line Interface (CLI), and infrastructure as code tools like AWS CloudFormation.

Amazon EKS has expanded support for Cilium as the Container Networking Interface (CNI) for Amazon EKS Hybrid Nodes. Cilium is a Cloud-Native Computing Foundation (CNCF) graduated project that provides core networking capabilities for Kubernetes workloads. Now, you can receive support from AWS for a broader set of Cilium features when using Cilium with Amazon EKS Hybrid Nodes including application ingress, in-cluster load balancing, Kubernetes network policies, and kube-proxy replacement mode. Kubernetes clusters require a CNI for connectivity between pods running in the cluster, but most Kubernetes applications require additional components, such as ingress controllers and load balancers, to serve and secure network traffic with other external systems or users. These additional capabilities are integrated features of Cilium, built on Cilium’s eBPF-powered networking and security. Now, Amazon EKS Hybrid Nodes users can receive support from AWS for Cilium’s Ingress and Gateway features, Border Gateway Protocol (BGP) Control Plane, Load Balancer IP Address Management (LB IPAM), kube-proxy replacement, and Kubernetes network policies. AWS supports the Amazon VPC CNI for Amazon EKS nodes in AWS Cloud, which is optimized for Amazon VPC networking with built-in features such as enhanced subnet discovery, Kubernetes network policies, and multiple network interfaces per pod.

Last up is news that Amazon EKS now supports deletion protection, helping you prevent accidental termination of your EKS clusters. When enabled, deletion protection requires explicit disablement before a cluster can be deleted, providing an additional safety control for critical environments. Deletion protection is turned off by default for all new and existing clusters. You can enable deletion protection during cluster creation or any time after. To delete a protected cluster, you must first disable deletion protection for the cluster and then proceed with the cluster deletion. This two-step verification process helps prevent unintended deletions that could result from automation errors or accidental commands, especially in environments where multiple users share cluster management responsibilities. Once enabled, any attempt to delete the cluster through the AWS Management Console, EKS APIs, AWS Command Line Interface (CLI), eksctl, or infrastructure as code tools like AWS CloudFormation will be blocked until deletion protection is disabled.

Videos of the month

AI Agents for Platform Engineers: EKS & OSS Upgrades via Chkk Upgrade Context MCP

Carlos Santana and Fawad Khaliq show how to give your AI coding agents (Amazon Q, Cursor, Claude Code, …) environment-aware context for upgrades and lifecycle management of OSS projects (Istio, Kafka, Keycloak, etc). They walk you through how it works, show a demo, and share what happens behind the scenes.

{% youtube SedzPt1rGGM %}

Deploy Secure Containers on Amazon EKS with Chainguard – AWS

In this recording, Sai Vennam from AWS details how Amazon EKS Auto Mode and Chainguard’s secure-by-default container images work together to streamline Kubernetes operations and strengthen your software supply chain. This session explores how EKS Auto Mode reduces the complexity of cluster management by automatically provisioning and scaling compute resources, while Chainguard’s hardened images help eliminate vulnerabilities by default — without sacrificing developer velocity. Sai shares actionable techniques for deploying production-ready workloads with greater efficiency and security, and wraps up with a hands-on demo to show you how easy it is to get started with these tools in tandem.

{% youtube BknFieQXSqA %}

Celebrating open source contributors

The articles and projects shared in this newsletter are only possible thanks to the many contributors in open source. I would like to shout out and thank those folks who really do power open source and enable us all to learn and build on top of what they have created.

So thank you to the following open source heroes: Sagi Tsofan, Adi Avni, Pini Dibask, Austin Groeneveld, Farooq Ashraf, Sumit Kumar, Devinder Singh, Ramesh Kumar Venkatraman, Jin Tan Ruan, Philipp Page, Lee Briggs, Curtis Rissi, Jason Janiak, Jobin Sukumaran, Boaz John, Shirish Kulkarni, Sukumar Sengottaiyan, Vivek Gobhil, Ryan Niksch, Mayur Shetty, Evan Grenda, Karan Singh, Kihyeon Myung, Sayan Chakraborty, Cindy Li, Akhil B, Harshana Nanayakkara, Joao Palma, Vishal Naik, Sumeet Tripathi, Yu-Ting Su, Jaydev Nath, David John Chakram, Sharmila Shanmugam, Will Leach, Nirupam Datta, Ryan Moore, Riccardo Freschi, Vishal Chaudhary, Ramesh Kandasamy, Brad Bebee, Forest Vey, Victor Servin, Jesse Butler, Erez Zarum, Henry Wang, Rod Wallace, Serge Shevchenko, Nick Aldridge, James Ward, Rashim Gupta, Emmanuel Isenah, Kenta Goto, kvendingoldo, Paul Santus, Vadym Kazulkin, Benjamen Pyle, Alejandro Velez, James Ward, Dhruv Thakur, Ryan Cormack and Jason Kao

Stay in touch with open source at AWS

Remember to check out the Open Source homepage for more open source goodness.

One of the pieces of feedback I received was to create a repo where all the projects featured in this newsletter are listed. Where I can hear you all ask? Well as you ask so nicely, you can meander over to newsletter-oss-projects.

Made with ♥ from DevRel