-
Notifications
You must be signed in to change notification settings - Fork 4
Better support forcing the write database instance #476
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
Converting back to draft to include CMS-1000. |
force_write_db util to work around database replication lag
MaciekBaron
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just two nit-picks, otherwise LGTM!
sanjeevz3009
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Apart from the comments left already. LGTM!
MaciekBaron
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Marking as "changes requested" as we still have those two nit-picks I left (sorry!)
|
@MaciekBaron Both should be fixed - I've just commented the relevant commits |
What is the context of this PR?
In #425, a race condition was fixed where data created would not be immediately available. This is likely because the replica instances had not synced the relevant data yet. The fix is to force the write instance to be used for the subsequent query.
Unfortunately, there's little which can be done about this lag. In the case of #425, the queries are already using
bulk_create/bulk_update, so they'll be done in a single operation, and should be synchronised at once. Therefore, trying to batch them in a transaction is unlikely to help.Since this pattern is likely to come up semi-often, I've written a small
force_write_db_for_querysethelper to make it clearer why.usingis being called. I've modified the dataset view to use theforce_write_db_for_querysethelper, but only when necessary, so that most cases can still use the replica instances. I've also addedforce_write_dbto work for an entire block, and added it to the publishing management commandsThe docs now reference these methods exclusively, rather than working around it with explicit
.usingcalls (which can be unclear) or using a transaction (which has a performance overhead).DEFAULT_DB_ALIASis the default fortransaction.atomic, so that can be removed.I've also added a test around nested transactions (ie savepoints), to confirm that definitely works.
How to review
Note that this PR contains both CMS-1000 and CMS-1001, due to the huge overlap in functionality.
Follow-up Actions
N/A