Cloudflare D1 and R2 consistency: keep records, assets and publication in sync
What happens when a D1 record is created but its R2 cover fails to upload? Designing reliable transfers, retries and publication integrity across services.
The failure begins between systems
When article metadata lives in D1 and photographs live in R2, a content update is no longer a single atomic database operation. A D1 transaction can succeed while an object upload fails; an object may also exist without a corresponding published record. Visitors then see broken covers, missing galleries or orphaned media. Define a clear difference between an uploaded object and an asset that is safe to publish. Draft, media-ready, verified and published states make these transitions observable. Publication should depend on verified media references rather than the hopeful assumption that every network operation finished. This makes integrity an explicit product requirement instead of a manual cleanup task.
An auditable write sequence
Allocate a stable content identifier first. Create a draft row in D1, upload the object to R2 under a key derived from content identity and a file digest, then verify its MIME type, size and availability. Record the validated media reference in D1 and publish only after required fields pass validation. If an intermediate step fails, resume from its recorded state instead of restarting blindly. D1 and R2 cannot be wrapped into one shared transaction, so compensating actions and idempotent retries must be designed explicitly. Keep enough metadata to distinguish an object that exists from a mapping that is committed and approved.
Retries must not duplicate content
A timeout does not prove the original upload failed. The operation may have succeeded while its acknowledgement was lost. Stable keys and hashes prevent every retry from generating a new object. Place unique constraints on content identifiers and media mappings, and verify them under concurrent requests. A simple application-level read-then-write sequence does not prevent two workers from racing. Use database uniqueness, appropriate transaction boundaries and bounded retry intervals. Record failures with cause and timestamp so operations can be resumed without repeated guesswork. Excessive retries cost both time and quota, even when the underlying task never progresses.
Publishing versus cache visibility
A successful write in the admin panel does not mean every visitor sees the new version immediately. Browser caches, Workers and edge caches have independent lifetimes. Measure database write completion, cache invalidation and public visibility separately. Versioned asset paths are usually more predictable than replacing the bytes behind a heavily cached URL. Instead of removing a current cover before publishing its replacement, keep the old valid version until the new asset and its database reference are ready. Monitor the delay from the editorial action to a confirmed public response. The optimization target is correct visible state, not just an HTTP success code.
Test failure paths and recovery
Simulate an R2 failure, a D1 rejection, several repeated submissions and simultaneous updates to the same record. Check that every published item has one expected, accessible cover mapping, while unattached objects are reported without automatic destructive cleanup. Review MIME type, response status, content digest where needed and the cache version shown to visitors. Rollback planning must cover both Worker code and the data written while it was running. A few extra validation queries introduce some cost; the benefit is fewer broken pages, less duplicate storage and faster root-cause analysis when an incident occurs.
Example: replacing a published cover
Consider an article or property whose cover must change. Instead of deleting the old image immediately, create a change operation with a stable identifier. Upload the new WebP to R2, validate it and then update the D1 cover reference. Keep the former image temporarily so delayed clients requesting the old version do not see a broken asset. Report unreferenced objects later through a separate, auditable cleanup workflow. If the connection fails midway, the system should know which cover is currently live and which uploaded file is awaiting publication. A retry can resume the missing stage without asking the editor to perform the entire procedure again. Versioned asset keys make this sequence easier to observe and verify.
Cost and capacity decisions
Downloading every image and rewriting every record each day can look simple, but it generates unnecessary object operations and database queries. Detect changes using content digests, ETags or update markers and refresh only the affected fields. Stored file metadata can also reduce repeated uploads under different names. Measure R2 reads and writes, D1 operations, bandwidth and Worker CPU rather than focusing only on how many jobs ran. Report unique successful content updates, validation failures and verified covers separately. A busy background agent is not necessarily a productive one. An automation that performs fewer operations while producing more correct records can be both cheaper and more reliable.
Comments
No comments yet. Be the first to contribute.
What might reader discussion look like?Illustrative
These names, comments and editor replies are illustrative, not real visitor posts. Interactions are not saved. Use the real form below to contribute.
Reading “Cloudflare D1 ve R2 ile veri tutarlılığı: kayıt, görsel ve yayın nasıl birlikte korunur?” raised this question for me: compatibility and debugging impact deserve review.
Editor note: Product claims and measured outcomes should be assessed separately. A small pilot with defined metrics is a useful first step.
We should also discuss maintenance cost. An easy rollout does not guarantee trouble-free upgrades.
I would start with a pilot and measurable success criteria rather than assuming a launch means proven value.
✎ Write a comment