RFC to determine the appropriate place for compute to run
Created by: courier-new
Currently, compute runs in the frontend
service. It will likely get too expensive to run replace
transformations from frontend
once we hit a significant number of files. This is also likely to become a general concern of compute, though replace
is the first and currently only example where it's relevant.
While we intend to build the search-and-replace prototype from frontend
, we should develop a plan for eventually moving compute out of it. Whether it requires its own new service, moves to a different existing service, or another resolution altogether is the question this RFC should seek to answer. The RFC should also consider the impact on other parts of the product that want to interact with compute (e.g. insights, notebooks), as well as deployment overhead if provisioning additional resources for compute is necessary.
Some discussion of ideas and their tradeoffs can be found in this thread.