Skip to content

Server-side execution of src action exec

Administrator requested to merge campaigns/remote-code into main

Created by: eseliger

Still super WIP

Things I've done so far:

  • A very basic version of the remote runner that pulls jobs and runs them in docker, streaming logs to the appendLog mutation and finally the patch and status using updateActionJob
  • GraphQL API definitions and stub API resolvers for all endpoints so the frontend can start again
  • actions query resolver filled with logic
  • New DB tables actions, action_executions, action_jobs
  • Mock UI for a running action

Things this approach could serve (I tick off what is actually implemented, not just drafted out somehow):

  • An API schema that allows for everything described below
  • A UI that shows all actions
  • A UI that shows an overview of an action execution
  • Saved searches can trigger action executions
  • A schedule for action executions can be defined as a CRON pattern
  • Uploading the workspace where the action lived on the local machine
  • Updating an action while maintaining previous runs
  • Streamed logs for all jobs, persisted individually for further inspection
  • Collecting the patches in the backend, automatically creating a campaign plan for those upon execution completion. (This will allow for the "Create campaign" and "Update campaign" buttons in the screenshot)
  • Attaching a campaign to an action, on completion, the campaign will be updated
  • Job timeouts (runner heartbeats, timeout starved jobs)
  • The possibility to still submit a campaign plan, when a subset of jobs failed
  • UI for associating a saved search with an action
  • UI for associating an action with a campaign

Things NOT included in this approach:

  • Rerunning an action job while maintaining the old jobs, currently it would need to be cleared
  • Cancellation of an execution is not implemented as part of the API/datamodel at this point

Tradeoffs:

  • To keep the runner save, I opted for wrapping the "command" type in a docker container as well, this probably requires some more thought and I defaulted it to "ubuntu" for now, but should definitely be configurable at least.
  • Running multiple jobs per repository is not possible, this is no regression over the previous version of actions, but to be considered in regards to monorepo support.

Learnings:

  • It would probably not be super hard getting something out for 3.14 that includes a subset of the proposed features. The runner somewhat works, but needs better error handling, concurrency control and logging. The API needs more fill-ins for resolvers but they are just tedious to type out not super difficult (at least most of them), and the frontend needs to be implemented (currently a mock).
  • Pulling jobs over having a complex Queue/Scheduler is probably perfectly fine for now, and a FIFO queue can be achieved by sorting the jobs ASC by ID

Note: the screenshot shows things that aren't implemented yet, but also shows thing not that are implemented, like log streaming and a proper diff view like known from the campaignplans page.

image

Update: gif

demo

Known bugs:

  • When a campaign is attached, Creating changeset: error in GraphQL response: Base ref must be a branch prevent the changesets from being created

Merge request reports

Loading