Automated Server Backups to S3 Storage With Duplicati

A backup job can finish successfully while leaving you unable to restore a working server. Missing database consistency, lost encryption credentials, and an untested recovery path are common reasons an offsite copy fails when it matters.

Duplicati can encrypt, deduplicate, and schedule backups to S3-compatible storage. This guide builds a practical workflow around it: prepare consistent source data, restrict remote access, plan transfer capacity, and prove recovery on a separate destination.

Establish recovery targets and transfer capacity

Start with recovery point objective: how much recent work could you afford to lose? Then define recovery time objective: how long can the service remain unavailable while data downloads and applications are rebuilt? These targets determine schedule, retention, and restoration priorities.

Use the Bandwidth & Transfer Time Calculator below to estimate the first upload and a full restore. Enter the data size and measured connection speed, then include an overhead allowance. Run upload and download estimates separately because your connection may be asymmetric.

The result is a planning estimate, not a backup duration guarantee. Duplicati also reads files, compresses data, hashes blocks, and makes remote API requests. Large numbers of tiny files, storage throttling, and limited CPU can dominate transfer time.

Define your backup scope explicitly: application configuration, persistent uploads, database exports, deployment manifests, and secret-recovery procedures. Exclude reproducible caches and temporary files after confirming they contain no unique data. A smaller, documented backup is easier to restore than an indiscriminate collection of unknown directories.

Prepare an isolated storage destination

Create a dedicated bucket or equivalent remote namespace with a least-privilege credential. Duplicati needs to list, read, write, and usually delete backup objects for its normal retention operations. Restrict those permissions to the backup bucket or prefix rather than granting control over the whole account.

Use HTTPS and verify the endpoint, region, and addressing requirements with your storage provider. “S3-compatible” does not promise identical behavior for every feature. The AWS S3 documentation explains the original service model, but another provider's endpoint and policy syntax may differ.

Separate operational access from administrative recovery. A credential stored on the protected server can be stolen if that server is compromised. Encryption helps protect confidentiality, but it does not prevent an attacker with deletion rights from destroying remote objects.

Bucket versioning, a separately administered replica, or immutable storage can reduce that risk. However, object locks and lifecycle policies must be tested with Duplicati's retention and compaction behavior. A policy that blocks required deletion can break maintenance; a lifecycle rule that expires active backup objects can make recovery impossible.

Record the bucket endpoint, region, prefix, and ownership in your recovery documentation. Store account recovery codes outside the server being backed up. Protect the storage account with multifactor authentication and alerts for unexpected administrative changes.

Produce consistent application data first

Duplicati backs up files; it does not automatically turn an actively changing database directory into a transactionally consistent backup. Export databases or use a database-aware snapshot method, then point the job at those prepared artifacts.

For MariaDB with InnoDB tables, this script creates an export and publishes it atomically after success. It assumes a restricted client option file already exists, enough local disk space, and a shell with pipeline failure checking.

#!/usr/bin/env bash
set -euo pipefail
umask 077
backup_dir=/srv/backup-input
install -d -m 700 "$backup_dir"
tmp=$(mktemp "$backup_dir/database.XXXXXX.sql.gz")
trap 'rm -f "$tmp"' EXIT
mariadb-dump --defaults-extra-file=/root/.my-backup.cnf   --single-transaction --quick --routines --events   --databases wordpress | gzip > "$tmp"
mv "$tmp" "$backup_dir/database.sql.gz"
trap - EXIT

The account needs appropriate export permissions. Avoid schema changes during this operation, and handle nontransactional tables separately. The MariaDB knowledge base documents dump behavior and compatibility constraints.

Schedule the export to finish before the backup job begins, or use a carefully tested pre-backup script. Ensure export failures stop the workflow and trigger an alert rather than silently backing up yesterday's file. Exclude the credentials file unless you have an explicit, secure secret-restoration plan.

Coordinate uploads with the database snapshot if records reference newly created files. A brief application write pause or a coordinated filesystem snapshot may be necessary when consistency across both sources matters.

Configure Duplicati and a sustainable schedule

Install a supported release using the Duplicati documentation and downloads. Run its service with only the local read permissions it needs. Protect its web interface with authentication and keep it inaccessible from the public internet.

For remote administration, bind the interface to localhost and forward it through SSH rather than opening its port broadly. This example assumes the service listens on its usual local port and that you have authorized SSH access.

ssh -N -L 8200:127.0.0.1:8200 admin@192.0.2.30

Open the forwarded address locally, then create a backup with client-side encryption enabled. Generate a strong passphrase and store it in an independent password manager or recovery vault. Neither the storage provider nor Duplicati can reconstruct a lost encryption passphrase for you.

Select the prepared export directory and required application files. Configure the S3-compatible backend, endpoint, bucket, and dedicated prefix. Test the connection, run the initial backup manually, and inspect warnings before enabling the schedule.

Choose retention deliberately. Keep enough historical versions to recover from unnoticed corruption or accidental deletion, while allowing room for changed blocks and maintenance operations. Backup volume size and upload concurrency should begin with conservative defaults; tune only after measuring resource use.

Return to the transfer time calculator when planning maintenance windows or changing retention. Deduplication can reduce future uploads, but a rewritten compressed database dump may still change substantially. Measure actual daily churn instead of assuming that every incremental run will be tiny.

Restore into a clean environment and monitor results

A restore drill should work without the original server's local database or filesystem. Keep a protected configuration export, encryption passphrase, and backend credentials separately. An export containing credentials is sensitive and belongs in encrypted recovery storage.

Restore a representative folder into an empty directory first. Compare expected filenames and content, then test database recovery in an isolated database instance. Never import a rehearsal dump over the live application database.

gzip -t /srv/restore/database.sql.gz
# Run only against an isolated recovery database server.
zcat /srv/restore/database.sql.gz |   mariadb --defaults-extra-file=/root/.my-recovery.cnf

A successful gzip integrity test proves only that the compressed stream is readable. Verify application recovery by connecting a staging application, checking record counts, opening recent uploads, and exercising a representative workflow with outbound notifications disabled.

Record elapsed download, import, and validation time. Include DNS changes, container image retrieval, and secret provisioning in the full recovery estimate. If the process exceeds your recovery target, revise the architecture before an incident forces the issue.

Monitor job age, warnings, storage growth, and restore-test age, not just a green completion indicator. Keep alerts outside the protected server so its disappearance is visible. The hosting operations collection provides related maintenance guidance.

Key takeaways

• Define recovery targets before choosing schedules and storage capacity.
• Export databases consistently; file backup alone does not guarantee a usable database.
• Protect encryption credentials separately and restrict remote storage permissions.
• Measure full restores, including application validation, not only backup upload duration.
• Treat restore drills and failure alerts as required parts of the backup system.