More AWS/GitHub notes

If you don't fancy hand-crafting YAML to set up your AWS infrastructure, Amazon offers the Cloud Development Kit. 

They've taken an interesting architectural decision. They've written the logic in JavaScript executed via node. But to make it a polyglot kit, they've added what are essentially bindings that run the node executable. I guess this makes the logic less likely to diverge between languages but it also means more hassle setting up your environment.

What it means is that the Java (or the supported language of your choice) does not talk to the AWS cloud directly. It generates files that you must then feed to cdk. This is different to Fabric8's Kubernetes client or  the docker-java library, both of which allow you to control the containerization in the JVM with no further tooling required.

[Aside: Terraform have a similar toolkit to CDK here but I gave up due to the lack of documentation].

CDK set up

AWS's Java CDK binding needs node in the PATH environment variable. Note, IntelliJ doesn't seem to put it into the PATH by default and I was getting the unhelpful "SyntaxError: Unexpected token ?" - a very unintuitive message. It appears that the Java implementation executes this node command using ProcessBuilder (see JsiiRuntime.startRuntimeIfNeeded) . 

You install the node runtime with 

npm install -g aws-cdk

You can initialize a Java project with:

cdk init app --language java

You then write your Java code to describe your environment using the libraries pulled in by the generated pom.xml. Knowing exactly what to do can be hard so you might want to look at some example on GitHub here.

When you're done, run your Java code. Then you can call cdk synth to generate the metadata and cdk deploy to deploy it and as if by magic, your S3 bucket etc is deployed in AWS! Note you must run this in the top-level directory of your Java project. Apparently, state is saved and shared between the Java build and the call to cdk.

All an act

You can run GitHub actions locally with act. This really helps with debugging. 

If you're .github YAML looks like this:

jobs:
  code:

Then run Act with something this:

act -n --env-file .github/pr.yml -j code -s AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID -s AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY

where the -j means which YAML block you're running.

Remove the -n if you want it to be a real run rather than a dummy run.

If you want to use secrets, you can just set them as environment variables. For instance:

        env:
          AWS_ACCESS_KEY: ${{ secrets.AWS_ACCESS_KEY_ID }}
        run: /root/.local/bin/poetry run python -m pytest integration_tests

will use your system's environment variable AWS_ACCESS_KEY_ID as the value of AWS_ACCESS_KEY that the integration tests use.