[seanlaff]

Im surprised there's still no "trace" view of pipeline execution, especially with how prevalent DAG pipelines have become. Somewhere on the internet I found this handy jq oneliner that will convert a pipeline into a format you can drag and drop into chrome://tracing/ to figure out where your bottlenecks are

  curl "https://${GITLAB_URL}/api/v4/projects/${GITLAB_PROJECT}/pipelines/${GITLAB_PIPELINE}/jobs?per_page=100&private_token=${GITLAB_TOKEN}" | jq 'map([select(.started_at and .finished_at) | {name: (.stage + ": " + .name), cat: "PERF", ph: "B", pid: .pipeline.id, tid: .id, ts: (.started_at | sub("\\.[0-9]+Z$"; "Z") | fromdate \* 10e5)}, {name: (.stage + ": " + .name), cat: "PERF", ph: "E", pid: .pipeline.id, tid: .id, ts: (.finished_at | sub("\\.[0-9]+Z$"; "Z") | fromdate \* 10e5)}]) | flatten(1) | .[]' | jq -s > "${GITLAB_PIPELINE}-trace.txt"

[dnsmichi]

Wow, thanks a lot for sharing. GitLab team member here.

Would it be ok for you if I add that command snippet into a blog post I am currently writing about Observability for Efficient DevSecOps Pipelines? Draft MR is in https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/34296 Thanks!

Regarding pipeline visibility and traces: I would love to see the same :-) I tested tracepusher with OpenTelemetry this week, and the timeline for CI/CD traces is a great start in Jaeger. Added a suggestion into https://gitlab.com/groups/gitlab-org/-/epics/5071#note_14582... where CI/CD Visibility is being worked on, with an update on GitLab support for traces in https://gitlab.com/groups/gitlab-org/-/epics/5071#note_14584...

[saurik]

FWIW, they had said that they didn't write it originally and had found it "somewhere on the Internet". Searching for "pid: .pipeline.id, tid: .id, ts" has turned up this, which might be the original source (if it is in fact the same script as I am on a phone and didn't 100% check):

https://gitlab.com/gitlab-org/gitlab/-/issues/236018#note_39...

[dnsmichi]

Oh, thanks, I read it wrong. Thanks for digging up the source, I want to be sure to give attribution where due.

Fantastic insights in the issue, next to the scripts. One could write a script that generates Mermaid charts in Markdown, and document the CI/CD infrastructure automatically - with CI/CD pipelines itself. Hmmm :)

[seanlaff]

Ah yup, looks like @saurik pin-pointed my original source (iirc I slightly tweaked the jq, but you get the idea). Please do spread the idea- I would love capability native like this in gitlab.

[carolinesimpson]

@seanlaff (and anyone else interested in having this built into gitlab), it would be great to upvote (thumbs up) the issue (https://gitlab.com/gitlab-org/gitlab/-/issues/236018) to help it get traction.

[michaelmior]

Note that for non-Chrome users, you can also view traces at https://ui.perfetto.dev/.

[stabbles]

I think you have to remove escaping: `\*` should be `*`