#codefresh - December 2018 | Slack Archive

codefreshArchived

201 messages

Archive: https://archive.sweetops.com/codefresh/

Nov 2018

December 2018

Jan 2019

Michal Matyjekalmost 8 years ago

@Michal Matyjek has joined the channel

Michal Matyjekalmost 8 years ago

Anyone is posting back from Codefresh to github? Looking for a complete solution to post back to github PR (not to commit) with previous comment deletion/removal. Ideally with some templating built-in 🙂

Use case: post helm diff, or twistlock scan results to PR - in a way there's only one comment for latest commit there.

Erik Osterman (Cloud Posse)almost 8 years ago

@Michal Matyjek yes, we have our github-commenter for that

Erik Osterman (Cloud Posse)almost 8 years ago

https://github.com/cloudposse/github-commenter

Erik Osterman (Cloud Posse)almost 8 years ago(edited)

it supports templated comments

Michal Matyjekalmost 8 years ago

does it overwrite?

Erik Osterman (Cloud Posse)almost 8 years ago

no - that would be a “feature request” 😃

Erik Osterman (Cloud Posse)almost 8 years ago

if someone on the gladly team wants to contribute that, we’ll promptly review and get it merged

Erik Osterman (Cloud Posse)almost 8 years ago

seems like it would be a nice feature, to be able to target a previous comment using some kind of regex

Michal Matyjekalmost 8 years ago

also getting the PR number from commit sha.... maybe using https://platform.github.community/t/find-pull-request-containing-specific-commit-hash/2827/6

Erik Osterman (Cloud Posse)almost 8 years ago(edited)

--upsert-comment="/some regex/"

Michal Matyjekalmost 8 years ago

@Daren ^^

Darenalmost 8 years ago

@Daren has joined the channel

Erik Osterman (Cloud Posse)almost 8 years ago

getting the PR number from commit sha.

Erik Osterman (Cloud Posse)almost 8 years ago

why?

Erik Osterman (Cloud Posse)almost 8 years ago

We use the CF_PR_NUMBER or something

Michal Matyjekalmost 8 years ago

oh is that in place?

Michal Matyjekalmost 8 years ago

i had no idea

Erik Osterman (Cloud Posse)almost 8 years ago

yea, sec

Michal Matyjekalmost 8 years ago

I think it did not always work for us or something - have not checked in months though

Erik Osterman (Cloud Posse)almost 8 years ago

CF_PULL_REQUEST_NUMBER

Erik Osterman (Cloud Posse)almost 8 years ago

we use it like this NAMESPACE=pr-${{CF_PULL_REQUEST_NUMBER}}-${{CF_REPO_NAME}}

Erik Osterman (Cloud Posse)almost 8 years ago

If you trigger a build manually, then you don’t get it

Michal Matyjekalmost 8 years ago

oh yeah - I think it's only if your trigger is on PR?

Erik Osterman (Cloud Posse)almost 8 years ago

it’s only available on synchronize

Erik Osterman (Cloud Posse)almost 8 years ago

or if you pass it manually

Michal Matyjekalmost 8 years ago

if you trigger on commit - you also dont I think

Erik Osterman (Cloud Posse)almost 8 years ago

but that’s maybe a tell-tale sign you don’t want to execute 😃

Michal Matyjekalmost 8 years ago

comment on PR - so that you can upsert previous commits

Erik Osterman (Cloud Posse)almost 8 years ago

since it’s happening outside of a PR

Erik Osterman (Cloud Posse)almost 8 years ago

you don’t want to do anything.

Michal Matyjekalmost 8 years ago

well if you push new commits to PR you do want it....

Erik Osterman (Cloud Posse)almost 8 years ago

that’s synchronize

Erik Osterman (Cloud Posse)almost 8 years ago

that’s supported by codefresh

erikalmost 8 years ago

Michal Matyjekalmost 8 years ago

hmmm... ok thanks!

Erik Osterman (Cloud Posse)almost 8 years ago

@Andriy Knysh (Cloud Posse) can you scope out this feature? see it it would be pretty easy to implement?

Andriy Knysh (Cloud Posse)almost 8 years ago

sure

Andriy Knysh (Cloud Posse)almost 8 years ago

override prev comment is what you are after, @Michal Matyjek?

Erik Osterman (Cloud Posse)almost 8 years ago

yea, but not just any previous comment

Erik Osterman (Cloud Posse)almost 8 years ago

the previous related comment

Erik Osterman (Cloud Posse)almost 8 years ago

so maybe use a regex input?

Michal Matyjekalmost 8 years ago

yeah

Erik Osterman (Cloud Posse)almost 8 years ago

and it would be like an “upsert”

Erik Osterman (Cloud Posse)almost 8 years ago

update or if not found insert

Michal Matyjekalmost 8 years ago

can be some magic string, since we can have multiple steps commenting to github from Codefresh - each comment can have separate magic string

Andriy Knysh (Cloud Posse)almost 8 years ago

ok let me see what could be done

Erik Osterman (Cloud Posse)almost 8 years ago

the other option @Daren mentioned was

Erik Osterman (Cloud Posse)almost 8 years ago

I like how k8s repos replace the comment

they actually delete, then recomment

Erik Osterman (Cloud Posse)almost 8 years ago

@Michal Matyjek do you have a preference?

Michal Matyjekalmost 8 years ago

I do not, it's OK to delete/recreate. Noone reads email here anyways except me and I am fine handling additonal github email notifications

Darenalmost 8 years ago

I like the delete/repost. Then the comment is inline with the action that triggered it

Darenalmost 8 years ago

Updating an existing comment can be confusing on PR with a lot of activity

Erik Osterman (Cloud Posse)almost 8 years ago

Yea, good point

Erik Osterman (Cloud Posse)almost 8 years ago

@Andriy Knysh (Cloud Posse) ☝️

Andriy Knysh (Cloud Posse)almost 8 years ago

👍️

Darenalmost 8 years ago

Since you can post html as the body, you might be able to use metadata tags as a means of labeling the comment for deletion

Darenalmost 8 years ago

Interesting, I looked a helm chart PR to see if I could spot how k8s does their automated comments. They are not posted as review comments, they show up as Issue comments

Darenalmost 8 years ago

I dont know the difference

Darenalmost 8 years ago

@Erik Osterman (Cloud Posse) how do you handle posting helm diffs to PRs when it contains secrets

Darenalmost 8 years ago

oh found it

Darenalmost 8 years ago

--suppress-secrets

Darenalmost 8 years ago

@Andriy Knysh (Cloud Posse) do you have some repos where codefresh is performing helm commands?

Andriy Knysh (Cloud Posse)almost 8 years ago

@Erik Osterman (Cloud Posse) was working on this one https://github.com/cloudposse/statup/blob/add-helmfile/codefresh/pull-request.yml#L93

Andriy Knysh (Cloud Posse)almost 8 years ago

the pipeline executes this helmfile https://github.com/cloudposse/statup/blob/add-helmfile/helmfile.yaml

Andriy Knysh (Cloud Posse)almost 8 years ago

this pipeline is for PR closed https://github.com/cloudposse/statup/blob/add-helmfile/codefresh/pull-request-closed.yml

Darenalmost 8 years ago

nice

Nikola Velkovskialmost 8 years ago(edited)

Morning, can someone assist me in setting up the pipeline in codefresh to push to ECR? I cannot seem to get it the push working.
I am getting the following error.
aused by Error: Failed to get registry credentials; caused by Error: Registry: '#userPassedEcr' could not be found

Erik Osterman (Cloud Posse)almost 8 years ago(edited)

(i’m headed to bed, but ping me in the morning if you don’t get it resolved)

Nikola Velkovskialmost 8 years ago

👍️ good night!

Nikola Velkovskialmost 8 years ago(edited)

Ok so I did a lot of small changes to the codefresh.yml and what I've noticed is some of them where the right ones but the step that fails is when pulling from codefreshes internal docker repo.


[SYSTEM] Error: Failed to calculate image name to push; caused by Error: Failed to pull image <http://r.cfcr.io/nikolavelkovski/nikola-dev-ecs@sha256:73f53ca80ea67f96259|r.cfcr.io/nikolavelkovski/nikola-dev-ecs@sha256:73f53ca80ea67f96259>
dd4e8ed66e99cde9a8da75c6c49ed1e4c1459985c9a4f; caused by Error: Failed to get registry credentials; caused by Error: Registry: '#userPassedEcr' could not be foun

Nikola Velkovskialmost 8 years ago

re-running this step works fine

Nikola Velkovskialmost 8 years ago

sometimes I need to re-run it 2 times.

Nikola Velkovskialmost 8 years ago

Here's a screenshot. I had to re-execute the push to ecr step 2 times in order to get it to pass without me doing any change.

Andriy Knysh (Cloud Posse)almost 8 years ago

@Nikola Velkovski can you share your pipeline? (remove all secrets 🙂 )

Nikola Velkovskialmost 8 years ago

there aren't any just a sec, so I've Alex on support who is investigating

Nikola Velkovskialmost 8 years ago


stage: push
title: Push image to ecr
type: push
description: Pushing to ecr
candidate: '${{build_image}}'
tag: 'codefresh-${{CF_REVISION}}'
image_name: nikola-dev-ecs
provider: ecr
accessKeyId: '${{AWS_ACCESS_KEY_ID}}'
secretAccessKey: '${{AWS_SECRET_ACCESS_KEY}}'
region: '${{AWS_ECR_REGION}}'

Nikola Velkovskialmost 8 years ago

This is the stage that it's failing with the cryptic error

Andriy Knysh (Cloud Posse)almost 8 years ago

Are you tagging the image and then push the branch and the tags for example? Maybe it fails to push some tags?

Nikola Velkovskialmost 8 years ago

from the error message it fails to pull the branch from the internal repo

Andriy Knysh (Cloud Posse)almost 8 years ago

can you show the pull step?

Nikola Velkovskialmost 8 years ago

sure wait

Nikola Velkovskialmost 8 years ago

hmm I've a build and a push step

Nikola Velkovskialmost 8 years ago

the pull is automatic

Nikola Velkovskialmost 8 years ago

upon commit

Andriy Knysh (Cloud Posse)almost 8 years ago

from GitHub?

Nikola Velkovskialmost 8 years ago

yes

Andriy Knysh (Cloud Posse)almost 8 years ago

ok sec

Nikola Velkovskialmost 8 years ago

Andriy Knysh (Cloud Posse)almost 8 years ago

try to add the clone step https://github.com/cloudposse/statup/blob/add-helmfile/codefresh/pull-request.yml#L10

Nikola Velkovskialmost 8 years ago

ok I will

Andriy Knysh (Cloud Posse)almost 8 years ago

@Michal Matyjek, @Daren

we improved https://github.com/cloudposse/github-commenter, let me test all the changes and update README with new examples, and we’ll release (today).
Added delete-comment-regex arg, so if provided, the module will try to find and delete all previous comments by the regex before creating a new one.

@Daren re:

Interesting, I looked a helm chart PR to see if I could spot how k8s does their automated comments. They are not posted as review comments, they show up as Issue comments. I dont know the difference

We went through all of that, and found many new details which we did not know before 🙂 (and it’s little bit convoluted).

Summary FYI:

GitHub API allows these types of comments:

1. Comments on repos/commits https://developer.github.com/v3/repos/comments
2. Coments on PR reviews https://developer.github.com/v3/pulls/reviews/#create-a-pull-request-review
3. Comments on PR file (at specified position) https://developer.github.com/v3/pulls/comments
4. Comments on issues https://developer.github.com/v3/issues/comments
5. Comments on PRs (in the global section, not on individual files) https://developer.github.com/v3/issues/comments

Since GitHub considers PRs as issues, #4 and #5 are basically the same (use the same API).

The new module release will support all five (Comment type: commit, pr, issue, pr-review or pr-file).

And all five will support the delete-comment-regex arg to delete the previous comments (if provided).

Andriy Knysh (Cloud Posse)almost 8 years ago

some screenshots FYI to see how all five look in the UI:

Andriy Knysh (Cloud Posse)almost 8 years ago

comment on commit:

Andriy Knysh (Cloud Posse)almost 8 years ago

Andriy Knysh (Cloud Posse)almost 8 years ago(edited)

comment on PR review (Comment #3):

Andriy Knysh (Cloud Posse)almost 8 years ago

Andriy Knysh (Cloud Posse)almost 8 years ago(edited)

Comments on PR file (at specified position) (Comment #5):

Andriy Knysh (Cloud Posse)almost 8 years ago

Andriy Knysh (Cloud Posse)almost 8 years ago(edited)

comment on issue (Comment #2):

Andriy Knysh (Cloud Posse)almost 8 years ago

Andriy Knysh (Cloud Posse)almost 8 years ago(edited)

comment on PR (in the global section) (Comment #4):

Andriy Knysh (Cloud Posse)almost 8 years ago

(all of those comments were created by the new module)

Nikola Velkovskialmost 8 years ago

@Andriy Knysh (Cloud Posse) it was solved, it turned out that if I remove the part where I explicitly state the push type and credentials and just use registry: nikola-dev-ecs it works

Nikola Velkovskialmost 8 years ago

¯\(ツ)/¯

Andriy Knysh (Cloud Posse)almost 8 years ago

nice

Andriy Knysh (Cloud Posse)almost 8 years ago

so yea, all those credentials should be settings in the ecr registry, not in pipeline steps

Nikola Velkovskialmost 8 years ago

well if one has let's say 50 ecr registries 🙂 then it gets interesting 🙂

Nikola Velkovskialmost 8 years ago(edited)

ok and another solution which I prefer was to quote the provider param

Nikola Velkovskialmost 8 years ago(edited)

provider: ecr == provider: 'ecr'

Nikola Velkovskialmost 8 years ago

@Andriy Knysh (Cloud Posse) thanks for the help!

mallenalmost 8 years ago

@mallen has joined the channel

Andriy Knysh (Cloud Posse)almost 8 years ago

@Daren @Michal Matyjek we just released https://github.com/cloudposse/github-commenter that supports delete-comment-regex argument

Andriy Knysh (Cloud Posse)almost 8 years ago

examples of this are here https://github.com/cloudposse/github-commenter#delete-comment-regex-example-1

Darenalmost 8 years ago

Thanks!

Erik Osterman (Cloud Posse)almost 8 years ago

(and all those args support ENV equivalents)

Andriy Knysh (Cloud Posse)almost 8 years ago

yep, here https://github.com/cloudposse/github-commenter#usage

Erik Osterman (Cloud Posse)almost 8 years ago

@dustinvb do you have a link handy for your claire image scanning?

Erik Osterman (Cloud Posse)almost 8 years ago

https://github.com/codefresh-contrib/cfstep-paclair

Erik Osterman (Cloud Posse)almost 8 years ago

@dustinvb is there some way to combine this with https://codefresh.io/codefresh-news/test-reporting/

Erik Osterman (Cloud Posse)almost 8 years ago

so we can bypass S3 dependency

dustinvbalmost 8 years ago

https://github.com/dustinvanbuskirk/example-voting-app/blob/master/worker/codefresh.yml#L19-L34

dustinvbalmost 8 years ago

Simple step to add.

dustinvbalmost 8 years ago

This is noted in the new issue I filed on the repository.

richwinealmost 8 years ago

@richwine has joined the channel

Erik Osterman (Cloud Posse)over 7 years ago

https://github.com/cloudposse/slack-notifier

Erik Osterman (Cloud Posse)over 7 years ago

we received a contribution which adds a SLACK_CHANNEL parameter so the same webhook can be reused

dustinvbover 7 years ago

Cool. Check out my bot? It handled gitops demos during KubeCon.

https://github.com/dustinvanbuskirk/cf-demo-bot

Erik Osterman (Cloud Posse)over 7 years ago

this is great!

Erik Osterman (Cloud Posse)over 7 years ago

I want to use this pattern to open PRs against cloudposse/packages to update packages on a cron using scheduled pipelines in codefresh

Erik Osterman (Cloud Posse)over 7 years ago

so much I want to do.... ugh

dustinvbover 7 years ago

I hear you, I have a todo to add GitHub releases into the bot and want to move from one bot to two bots and make the commit items a bit more flexible maybe format them in some way.

dustinvbover 7 years ago(edited)

New CLI Feature: https://codefresh-io.github.io/cli/validation/validate-pipeline-yaml/

dustinvbover 7 years ago

Erik Osterman (Cloud Posse)over 7 years ago

@Igor Rodionov

Igor Rodionovover 7 years ago

@Igor Rodionov has joined the channel

Igor Rodionovover 7 years ago

nice, but I can not now imaging where to use this

Erik Osterman (Cloud Posse)over 7 years ago

We should have this as a step in your pipeline as code stuff

Erik Osterman (Cloud Posse)over 7 years ago

This will validate the rendered templates of the Codefresh yaml are valid

dustinvbover 7 years ago

Team is working on a --local option for using local machine for runner for Codefresh builds as well. I'll post that here as soon as available that will probably be more widely used.

Erik Osterman (Cloud Posse)over 7 years ago(edited)

oh that's sweet

Michal Matyjekover 7 years ago

validate - finally....

Michal Matyjekover 7 years ago

thanks!

Erik Osterman (Cloud Posse)over 7 years ago

@Michal Matyjek you should see the pipeline creation for Codefresh @Igor Rodionov wrote

Erik Osterman (Cloud Posse)over 7 years ago

Cicd creation of Codefresh repos wired up to pipelines

Erik Osterman (Cloud Posse)over 7 years ago

So you don’t need to login to Codefresh to even setup a new service

Michal Matyjekover 7 years ago

part of me says "that's awesome" but part is like "nooooo...." 😄

Erik Osterman (Cloud Posse)over 7 years ago

haha, how come?

Erik Osterman (Cloud Posse)over 7 years ago

i love the idea that setting up a new repo is all in code

Michal Matyjekover 7 years ago

versioning I guess

Erik Osterman (Cloud Posse)over 7 years ago

in fact, i even like the idea of creating the repos that way too

Michal Matyjekover 7 years ago

yeah

Erik Osterman (Cloud Posse)over 7 years ago

so that they are setup with branch protections

Michal Matyjekover 7 years ago

that makes sense

Michal Matyjekover 7 years ago

but to me feels like something that should be out-of-the-box from Codefresh, not separate

Erik Osterman (Cloud Posse)over 7 years ago

ya…. kind’a agree

Michal Matyjekover 7 years ago

even basic stuff - like autocreating pipelines by the nature of having file named codefresh.yml in the root of the repo....

Michal Matyjekover 7 years ago

which I guess the above is also doing 🙂

Erik Osterman (Cloud Posse)over 7 years ago

(or a codefresh/ directory of pipelines)

Michal Matyjekover 7 years ago

that would be .codefresh/ 😄 - this is what we standardized on at Gladly, so everyone please use that 😄

Michal Matyjekover 7 years ago

on why "noooo...." - is we're having discussions internally on how much to abstract from the dev team - and there are some good arguments to not to abstract (would apply to pipeline creation, but also monochart in helm etc.)

Michal Matyjekover 7 years ago

there are also good arguments to abstract - for patterns, ability to easily inject stuff etc.

Erik Osterman (Cloud Posse)over 7 years ago

yea, i see what you mean.. getting farther away from understanding how it works

joshmyersover 7 years ago(edited)

A cli tool so you can run pipelines locally is awesome. Wercker (https://github.com/wercker/wercker) had a cli for local which was nice to use before they got bought

dustinvbover 7 years ago

New option --yaml to run Codefresh pipeline using the YAML file from your local machine.

https://codefresh-io.github.io/cli/pipelines/run-pipeline/

dustinvbover 7 years ago

New option --local to run Codefresh pipeline using local Docker daemon now available see link above for option on pipeline run from CLI.

Erik Osterman (Cloud Posse)over 7 years ago

wow, kick ass! moving fast……

Erik Osterman (Cloud Posse)over 7 years ago

@Igor Rodionov what do you think about the local option?

Erik Osterman (Cloud Posse)over 7 years ago

@dustinvb does it support shared configurations?

Igor Rodionovover 7 years ago

Hello

dustinvbover 7 years ago

I haven't had a chance to work with it yet and check out the shared configurations or other options. I believe right now the only thing not supported is localized volume caching.

dustinvbover 7 years ago

That is due out next week I believe.

Igor Rodionovover 7 years ago

how it works with shared configurations?

Igor Rodionovover 7 years ago

and let's say - k8s integration

dustinvbover 7 years ago

Ah like getting the context?

Erik Osterman (Cloud Posse)over 7 years ago

--local = local development

Igor Rodionovover 7 years ago

yea

Erik Osterman (Cloud Posse)over 7 years ago

oh, access to k8s integrations

dustinvbover 7 years ago

I'll try it real fast, I've been a bit limited on time.

Erik Osterman (Cloud Posse)over 7 years ago

(not urgent on our end)

dustinvbover 7 years ago

You have sparked my interest Sir!

dustinvbover 7 years ago

It all appears to work fine

dustinvbover 7 years ago

I'll post my command and output here.

dustinvbover 7 years ago

So appears to work flawless with Kube Context etc.

dustinvbover 7 years ago

YAML for this pipeline ☝️ is located here: https://github.com/dustinvanbuskirk/example-voting-app/blob/master/.codefresh/codefresh-dvts.yml

Igor Rodionovover 7 years ago(edited)

I'm very excited to test this

Erik Osterman (Cloud Posse)over 7 years ago

yea, this is super dope

Erik Osterman (Cloud Posse)over 7 years ago

@Florian Drescher talk to @Valdemir if you want a demo

Florian Drescherover 7 years ago

@Florian Drescher has joined the channel

Valdemirover 7 years ago

@Florian Drescher I d be more than happy to help. When would be most convenient for us to meet?

Michal Matyjekover 7 years ago

@dustinvb one thing that would be great to have is being able to pause execution - for pipeline debugging purposes - add a parameter that takes step name and execution stops at this step, so that you can connect to the container and investigate things in the given state

Erik Osterman (Cloud Posse)over 7 years ago

yea, like a break point?

Erik Osterman (Cloud Posse)over 7 years ago

ohhhh yea, that is the BEST thing with circle

Erik Osterman (Cloud Posse)over 7 years ago

love being able to exec into a container

Erik Osterman (Cloud Posse)over 7 years ago

(dustin is on vacay)

Erik Osterman (Cloud Posse)over 7 years ago

@Michal Matyjek they said you can open feature requests here: https://github.com/codefresh-io/cli/issues

Erik Osterman (Cloud Posse)over 7 years ago

➕️ 1 for adding that

Michal Matyjekover 7 years ago

https://github.com/codefresh-io/cli/issues/254

Erik Osterman (Cloud Posse)over 7 years ago

https://github.com/codefresh-io/cli/issues/255

November 2018 Browse by date January 2019