Skip to content

RFC to determine the appropriate place for compute to run

Created by: courier-new

Currently, compute runs in the frontend service. It will likely get too expensive to run replace transformations from frontend once we hit a significant number of files. This is also likely to become a general concern of compute, though replace is the first and currently only example where it's relevant.

While we intend to build the search-and-replace prototype from frontend, we should develop a plan for eventually moving compute out of it. Whether it requires its own new service, moves to a different existing service, or another resolution altogether is the question this RFC should seek to answer. The RFC should also consider the impact on other parts of the product that want to interact with compute (e.g. insights, notebooks), as well as deployment overhead if provisioning additional resources for compute is necessary.

Some discussion of ideas and their tradeoffs can be found in this thread.