From f5691dbb5e79b8f97a23ddbcffdf071888149a4b Mon Sep 17 00:00:00 2001 From: rhymes Date: Mon, 19 Apr 2021 16:11:10 +0200 Subject: [PATCH] [15 min fix] Document data update scripts best practices (#13432) * Add data update scripts best practices * Remove leftover counter --- docs/backend/data-update-scripts.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/docs/backend/data-update-scripts.md b/docs/backend/data-update-scripts.md index b062685f9..8e78d5f8f 100644 --- a/docs/backend/data-update-scripts.md +++ b/docs/backend/data-update-scripts.md @@ -79,3 +79,25 @@ DataUpdateScripts are also run automatically when a production deploy goes out. However, to ensure the new code they need to use has been deployed we use a [`DataUpdateWorker`](https://github.com/forem/forem/blob/main/app/workers/data_update_worker.rb) via Sidekiq and set it to run 10 minutes after the deploy script has completed. + +## Best practices + +### Working with large collections of rows + +From time to time, scripts need to operate on a large amount of rows; in those +cases we encourage: + +- adding explicit logging to the script +- reversing the order, to start processing the most recent records first + +For example: + +```ruby +def run + Article.find_each(order: :desc).with_index do |article, index| + Rails.logging.info("...") if index % 1000 == 0 # this will log every 1000 articles + + article.save + end +end +```