Agile Java Man

This proved surprisingly hard but the take-away points are: A Kubernetes service account must be given permissioned to access AWS infra stru...

This is how you get Snowflake to talk to your AWS real estate. Before we start, get your AWS account ID with: aws sts get-caller-identity  T...

A problem I was having when spinning up a K8s cluster and then trying to deploy my own Polaris was that the pod stuck in the  Pending stat...

I've found that managing Spark clusters in Kubernetes is far easier using the Spark Operator. Here are some commands that helped me diag...

Setting up a 3 node Kubernetes cluster in AWS is as simple as: eksctl create cluster --name $CLUSTERNAME --nodes 3 but this really hides a h...

Iceberg has a Kafka Connect component that ensures exactly one semantics despite there being two transactions for the two systems (Kafka and...

This is a high-level overview of my work creating a Spark cluster running in a managed AWS Kubernetes cluster (EKS) giving Spark the permiss...