#airship - January 2019

@Bryan has joined the channel

Welcome @Bryan what brought you here ?

hey @Maarten van der Hoef, back in the land of the living?

yap 🙂

happy NY

qq: can you remind me why the airship scheduled tasks use lambda rather than just pure cloudwatch events?

the lambda checks for the current task definition, so after updating the ECS service it will stick with what is currently running.

normal scheduled tasks are configured with a certain task definition, so after updating the service they would refer to a non-active taskdef

have a better idea ?

right! thanks. will ponder it

@Maarten van der Hoef so we are belatedly looking at moving everything to newer airships with the lambda lookup/task definition selector; but this means creating O(N) of all of the resources involved - aws_iam_role.lambda_lookup, aws_iam_role_policy.lambda_ecs_task_scheduler_policy (even if no task scheduling is used), aws_iam_role_policy.lambda_lookup_policy, aws_lambda_function.lambda_lookup, as well as various data sources. ideally we would have O(1), e.g. just one lambda that can be used for multiple defs with the necessary parameters passed in. any thoughts on this?

Cool.. For the purpose of speeding up I assume ? Have you thought of creating seperate env's per application, this really works well, also limits blast radius.

1. For the lambda lookup we could create a function_arn param which overrides and disables the current one.
2. lambda_ecs_task_scheduler_policy can be fixed ..

Partly speeding up but also just general cleanliness/KISS - if we have 100 services, then we don't really want to have 100 lambdas "just" to help with their deployment, if we can do the same thing with 1

@puru has joined the channel

So checked out Traefik a bit, looks promising and could be cool for a reference architecture.
It needs a K/V store for Cluster mode, and DynamoDB support is currently not merged yet in the library it uses https://github.com/abronan/valkeyrie so that's that.

Without cluster mode every instance of traefik would hit the aws api's for changes, which doesn't scale

Thanks Erik! That got time, it is too expensive to do anything with it.

@Maciek Strömich has joined the channel

I think if Airship has Traefik with something like this, I think it can be a more complete project, what do you guys think ?
https://github.com/containous/traefik/issues/4363

0.8.7: Label support (#38)
Label support
Restructure input blocks

Thanks @Maarten van der Hoef for this great project and outstanding docs.

Here is a small problem that I encountered with getting started guide. When I tried to apply everything at once (including ECS cluster and service) this errors occur:

* module.fargate_service.module.alb_handling.aws_route53_record.record_alias_a: aws_route53_record.record_alias_a: value of 'count' cannot be computed
* module.fargate_service.module.alb_handling.aws_lb_listener_rule.host_based_routing_custom_listen_host_redirect_to_https: aws_lb_listener_rule.host_based_routing_custom_listen_host_redirect_to_https: value of 'count' cannot be computed
* module.fargate_service.module.alb_handling.aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host: aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host: value of 'count' cannot be computed
* module.fargate_service.module.alb_handling.aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host_cognito_auth: aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host_cognito_auth: value of 'count' cannot be computed
* module.fargate_service.module.alb_handling.aws_lb_listener_rule.host_based_routing_custom_listen_host: aws_lb_listener_rule.host_based_routing_custom_listen_host: value of 'count' cannot be computed
* module.fargate_service.module.alb_handling.aws_route53_record.record: aws_route53_record.record: value of 'count' cannot be computed

But if you plan and apply in exact moments that you described in the guide it works fine.

Now I'm struggling with adding CI/CD for it. I'm trying to use https://github.com/cloudposse/terraform-aws-ecs-codepipeline but have no idea how to pass variables to buildfile.yml. Have you got any clues?

Hi @Lukasz German I've created a ticket for that, will have a look what can be done there. I'll have a look on Monday to see if we can combine it with terraform-aws-ecs-codepipeline

0.8.8: Add more output variables (#42)
More outputs

hey all! im fairly new to terraform so i really enjoyed reading about the airship modules for setting up ECS 🙂

my only question is… im currently wanting to testdrive how the airship modules creates the ECS cluster and service however it looks like in the getting started guide i am forced to own a domain as a requirement. is there any way i can get through the getting started guide WITHOUT having to setup a route53 host and SSL?

Hi @johnbeans! It's a bit hard without as the ALB does the 'routing' based on the hostname

hey @Maarten van der Hoef is this a requirement of AWS or airship’s module? please note im still somewhat new to using AWS with ALB and ECS so pardon my ignorance if its a silly question 🙂

The getting started was made to utilize the ALB. But for the ecs module it's not necessary.

do you want to reach the docker service from the outside ?

yes i do

It's really much simpler to get a (sub) domain and work from there, but to have something to work without.
for use with ALB you do need the ALB https listener as it's kind of mandatory. What you can do is to supply the ALB with a self signed certificate.

Within load_balancing_properties you can set route53_record_type to none, so you don't need to provide a route53_zone_id.

You can add the dns name of the alb to the custom_listen_hosts, this make sure a rule is made so that traffic to the dns name of the alb is forwarded to the ecs task.

thanks maarten 🙂 it sounds like ill be better off just getting a domain that i can sandbox with 🙂

so i got the getting started guide working 🙂 very smooth!

i noticed that if i want to change the desired_capacity after initially following the getting started guide, it does not actually result in expected behaviour. instead, it creates a new empty cluster called “fargate” and with the existing service “demo-web” it just creates a copy of the existing task definition and deploys it without actually changing the number of tasks… am i doing something wrong or using the incorrect property in the module? id ultimately like to increase the desired count from 1 to 2

Hi @johnbeans set the min capacity to two and you will have it with two. Desired count is ignored in the lifecycle of the ecs_service, meaning it can only be used for the bootstrap.
The reason for this is that in an environment where scaling changes the desired_count, one does not want the terraform run to set the desired_count back to the initial value.

I just got airship up and running for first time. with the demo-web example. great!

ok now, uhm.. how do I do a rolling deploy? lol

looking at codedeploy now