#codefresh - August 2019 | Slack Archive

codefreshArchived

68 messages

Archive: https://archive.sweetops.com/codefresh/

Jul 2019

August 2019

Sep 2019

dustinvbover 6 years ago

@Erik Osterman (Cloud Posse) Do you have an example step where you utilize Helmfile in a Codefresh pipeline?

dustinvbover 6 years ago

Have someone who would like to see how this is used in a Codefresh pipelin

Erik Osterman (Cloud Posse)over 6 years ago

yes, but not maintained/current.

Erik Osterman (Cloud Posse)over 6 years ago

https://github.com/cloudposse/testing.cloudposse.co/tree/master/codefresh/helmfile

dustinvbover 6 years ago

There is nothing your team uses currently as part of Codefresh deploy with a helmfile in a pipeline? I thought this was something you were doing regularly for a majority of your Helm deploys.

Erik Osterman (Cloud Posse)over 6 years ago

oh that

Erik Osterman (Cloud Posse)over 6 years ago

Yes, we use this pattern: https://github.com/cloudposse/example-app/tree/master/deploy

Erik Osterman (Cloud Posse)over 6 years ago

https://github.com/cloudposse/example-app/blob/master/codefresh/deploy.yaml

Erik Osterman (Cloud Posse)over 6 years ago(edited)

but the ctl command there is optional. we have an inline approach we use with our latest pipelines.

Erik Osterman (Cloud Posse)over 6 years ago

instead of that deploy_helmfile, we use a step more like

Erik Osterman (Cloud Posse)over 6 years ago


deploy_helmfile:
    title: Deploy with helmfile
    stage: Deploy
    image: "${{TOOL_DOCKER_REPO_PREFIX}}/${{AWS_ENV_NAME}}:latest"
    working_directory: /conf/
    environment:
      - 'KUBECONFIG=${{CF_KUBECONFIG_PATH}}'
      - 'REPO_ROOT=${{CF_VOLUME_PATH}}/${{CF_REPO_NAME}}'
      - 'APP_HOST=${{APP_HOST_DOMAIN}}'
    shell: bash
    commands:
      # Announce the release version
      - echo "* Preparing to deploy ${RELEASE_NAME} with image ${IMAGE_NAME}:${IMAGE_TAG}"
      - echo "*     with flavor \"$FLAVOR\" into namespace ${RELEASE_NAMESPACE}"
      - kubectl config use-context "${KUBE_CONTEXT}"
      - cd $(conf-directory helmfiles) && echo Working directory is now $PWD
      - direnv exec . helmfile --namespace="${RELEASE_NAMESPACE}" --environment="${FLAVOR}"
        -f "${REPO_ROOT}/deploy/helmfile.yaml" sync
    when:
      steps:
      - name: ask_for_permission
        on:
        - approved

Alex Siegmanover 6 years ago(edited)

Seems that pending-approval steps don't parse variables in the title field. I'd love to be able to customize that text for better UX on reused pipelines. Anyone done something similar? Trying to do this:


  ask_for_permission:
    type: pending-approval
    title: Deploy release of ${{APP_NAME}}:${{IMAGE_TAG}} to ${{FLAVOR}}?
    stage: Prepare

But I get the text seen in title with no substitution for the variables. Is this a feature request for the codefresh folks?

Erik Osterman (Cloud Posse)over 6 years ago

@Alex Siegman totally agree

Erik Osterman (Cloud Posse)over 6 years ago

We've submitted a similar feature request

Erik Osterman (Cloud Posse)over 6 years ago

I think the more that do it, the better

Erik Osterman (Cloud Posse)over 6 years ago

Go to support.codefresh.io to add your vote for this

Erik Osterman (Cloud Posse)over 6 years ago

I know @Jeremy G (Cloud Posse) requested it too

Jeremy G (Cloud Posse)over 6 years ago

@Jeremy G (Cloud Posse) has joined the channel

Alex Siegmanover 6 years ago

Feature request sent

btaiover 6 years ago

for codefresh, sometimes my pipeline will pull a previously built image to use as the cache and other times it will say the following:

Previous built image: <http://r.cfcr.io/org/repo_name@sha256:hash|r.cfcr.io/org/repo_name@sha256:hash> exists on the daemon and will be us
ed as cache for the build

btaiover 6 years ago

so im wondering if you have many devs working concurrently on the same repo (in different branches) then does that mean that it’s more likely your pipeline will need to pull the image (adds ~1.5-2min for a 2.5GB image) as opposed to already having the image existing on the daemon

Erik Osterman (Cloud Posse)over 6 years ago(edited)

yes, I think it depends on if the build hits a worker (and thus different daemon) with the image cached.

Erik Osterman (Cloud Posse)over 6 years ago

@Igor Rodionov spent a lot of time looking into this recently

btaiover 6 years ago

😕

btaiover 6 years ago

when you say worker

btaiover 6 years ago

you mean k8s worker i assume that codefresh is running on?

Erik Osterman (Cloud Posse)over 6 years ago

so behind the scenes they run Docker-in-Docker (for security reasons).

Erik Osterman (Cloud Posse)over 6 years ago

it spawns a dind on demand

Erik Osterman (Cloud Posse)over 6 years ago

i thin their SaaS product works slightly different than their enterprise product though

Erik Osterman (Cloud Posse)over 6 years ago

best thing is to go to <http://support.codefresh.io|support.codefresh.io> and open issue

Erik Osterman (Cloud Posse)over 6 years ago

they'll explain better than me 😃

Erik Osterman (Cloud Posse)over 6 years ago

@dustinvb is on vacay i think

btaiover 6 years ago

is dustin the only cf guy on sweetops?

btaiover 6 years ago

i was under the impression each pipeline had a persistent volume that would store many docker image layers

Erik Osterman (Cloud Posse)over 6 years ago

this is just an informal community driven channel. @Dan Garfield and @Francisco Cocozza are here too, but may not be SME.

Erik Osterman (Cloud Posse)over 6 years ago

And @Oriel Belzer for enterprises sales questions 😃

btaiover 6 years ago(edited)

no worries, i wasnt planning on reaching out to anyone directly but if they read this channel and have comments, im not opposed 😛

btaiover 6 years ago

i guess im wondering how I can maximize the image existing on the daemon so the build doesnt spend an extra 1.5~2min pulling the image first

Erik Osterman (Cloud Posse)over 6 years ago

yea, we've been bit by this as well. for on prem, we reduced the number of build nodes and increased the size of the nodes.

Erik Osterman (Cloud Posse)over 6 years ago(edited)

not sure what to do for the hosted version though to improve cache hits (or if its even the problem we had)

btaiover 6 years ago

that makes sense

btaiover 6 years ago

for on prem

Erik Osterman (Cloud Posse)over 6 years ago

#office-hours starting now! https://zoom.us/j/508587304

btaiover 6 years ago

is it possible to have a pipeline that listens on a github org? allowing it to create new projects/pipelines based on new repo creation?

btaiover 6 years ago

woops forgot @Erik Osterman (Cloud Posse) already asked this above ^^ 😅

Erik Osterman (Cloud Posse)over 6 years ago

haha

Erik Osterman (Cloud Posse)over 6 years ago

😃

Erik Osterman (Cloud Posse)over 6 years ago

yea, not yet.

Erik Osterman (Cloud Posse)over 6 years ago

however, using repo templates might not be a bad start

Erik Osterman (Cloud Posse)over 6 years ago

also, in the repo template, include a pipeline-spec

Erik Osterman (Cloud Posse)over 6 years ago

and a makefile

Erik Osterman (Cloud Posse)over 6 years ago


version: '1.0'
kind: pipeline
metadata:
  name: ourcompany/example-app/destroy-pull-request
  isPublic: false
  project: etan-tolling/example-app
  deprecate:
    repoPipeline: true
    implicitGitCloneService:
      context: github
      scm:
        provider: github
        owner:
          name: ourcompany
        name: example-app
spec:
  triggers:
    - name: ourcompany/destroy-pr
      type: git
      repo: ourcompany/example-app
      events:
        - pullrequest.closed
        - pullrequest.unlabeled
      pullRequestAllowForkEvents: false
      commentRegex: /.*/gi
      branchRegex: /.*/gi
      branchRegexInput: regex
      provider: github
      disabled: false
      options:
        noCache: false
        noCfCache: false
        resetVolume: false
      context: CF-default
      contexts: []
      variables: []
  steps: {}
  stages: []
  variables: []
  contexts:
    - Slack
    - GitHub
  specTemplate:
    location: git
    repo: ourcompany/example-app
    path: ./codefresh/destroy-pull-request.yml
    context: github

Erik Osterman (Cloud Posse)over 6 years ago

that's an example spec

Erik Osterman (Cloud Posse)over 6 years ago


codefresh create pipeline -f destroy-pull-request.yml

Andriy Knysh (Cloud Posse)over 6 years ago

A Codefresh pipeline for creating pipelines

Andriy Knysh (Cloud Posse)over 6 years ago

https://codefresh.io/devops-tutorial/programmatic-creation-codefresh-pipelines-part2/

btaiover 6 years ago

^ that requires a monorepo 😕

rssover 6 years ago

Github services are partially degraded
Aug 22, 10:14 UTC
Investigating - GitHub informs:
Issues, PRs, Dashboard, Projects, Notifications are partially degraded

rssover 6 years ago

Github services are partially degraded
Aug 22, 11:11 UTC
Update - GitHub reports a major outage on
Issues, PRs, Dashboard, Projects, Notifications servicesAug 22, 10:14 UTC
Investigating - GitHub informs:
Issues, PRs, Dashboard, Projects, Notifications are partially degraded

rssover 6 years ago

Github services are partially degraded
Aug 22, 12:23 UTC
Resolved - This incident has been resolved.Aug 22, 11:11 UTC
Update - GitHub reports a major outage on
Issues, PRs, Dashboard, Projects, Notifications servicesAug 22, 10:14 UTC
Investigating - GitHub informs:
Issues, PRs, Dashboard, Projects, Notifications are partially degraded

rssover 6 years ago

Bitbucket has reported an incident on their service
Aug 27, 17:23 UTC
Investigating - Incident details:
Bitbucket commits API high response time and timeouts
This incident affects: API, SSH, Git via HTTPS, Mercurial via HTTPS, and Pipelines.

More info: https://bitbucket.status.atlassian.com/incidents/d1nyzp56331y

rssover 6 years ago

Bitbucket has reported an incident on their service
Aug 28, 08:47 UTC
Resolved - The incident has been resolved and all systems are operational.Aug 27, 17:23 UTC
Investigating - Incident details:
Bitbucket commits API high response time and timeouts
This incident affects: API, SSH, Git via HTTPS, Mercurial via HTTPS, and Pipelines.

More info: https://bitbucket.status.atlassian.com/incidents/d1nyzp56331y

btaiover 6 years ago

for those using direnv in codefresh, do you have a seperate file that you rename to .envrc when the pipeline runs? How do you handle committing that to the repo for the pipeline and having a local copy

Erik Osterman (Cloud Posse)over 6 years ago

https://github.com/cloudposse/testing.cloudposse.co/blob/master/codefresh/terraform/pipeline.yml#L76

Erik Osterman (Cloud Posse)over 6 years ago

I don’t follow your question

Erik Osterman (Cloud Posse)over 6 years ago

But this is how we did it

rssover 6 years ago

Codefresh build clusters networking issues
Aug 31, 14:39 UTC
Investigating - Amazon has reported connectivity issues affecting some instances.
They are currently working on investigating the issues
More information can be found on
https://status.aws.amazon.com/

rssover 6 years ago

Codefresh build clusters networking issues
Aug 31, 20:47 UTC
Resolved - AWS Incident is no longer affecting CF.Aug 31, 14:39 UTC
Investigating - Amazon has reported connectivity issues affecting some instances.
They are currently working on investigating the issues
More information can be found on
https://status.aws.amazon.com/

July 2019 Browse by date September 2019