Skip to main content

Retire MongoDB 4.0 after cutover

After cutover, the application runs on MongoDB 8.0 under the temporary names <stack>_db8 or <stack>_mongoN-8. The retire command gives the canonical names back to the MongoDB 8.0 data, so the deployment matches a new MongoDB 8.0 installation. Run it after the application has run on MongoDB 8.0 long enough for you to trust it.

warning

On a three-node cluster after retire, the s1 db, s1 set-password, and s1 database-status commands fail, because they look for the replica set secureone, but the replica set's name is secureone8. The application still works.

  • To open a shell, use the manual method in Open a shell manually, and use the third connection string.
  • s1 database-status reports nothing useful here. Use the shell to inspect the database.
  • To change the password of the built-in secureone administrator account, contact Netwrix Support. The workaround for the shell doesn't cover s1 set-password.

The retire command never deletes the MongoDB 4.0 data. It renames that data aside, so a rollback remains possible until you remove the archive.

What retire changes​

ItemSingle nodeThree-node cluster
Services<stack>_db runs MongoDB 8.0 on the canonical data path. The command removes <stack>_db8.<stack>_mongo1, <stack>_mongo2, and <stack>_mongo3 run MongoDB 8.0. The command removes the -8 services.
Data pathMongoDB 8.0 data moves to /secureone/data/db.The same, on each node, one member at a time.
MongoDB 4.0 dataRenamed to /secureone/data/db.mongo4.<timestamp>.Renamed beside the canonical path on each node.
Replica setNot applicable.Stays secureone8. The tool renames hosts, not the replica set.
/secureone/s1.envPins MONGO_VERSION. Removes the DB, REMEDIANT_DB, and DB_REPLICA_COUNT settings that cutover added, so the stack defaults apply.Pins MONGO_VERSION, MONGO_REPLICA_SET, and DB (to the canonical connection string). Removes the MONGO1_REPLICA_COUNT, MONGO2_REPLICA_COUNT, and MONGO3_REPLICA_COUNT settings.
docker-stack.mongo-state.ymlRemoved.Removed.

For a cluster, the tool processes secondary members first and the primary last, so the replica set keeps a majority throughout.

Before you begin​

  • Complete Cut over to MongoDB 8.0, and verify the result. See Verify and maintain the deployment after cutover.
  • Confirm that an independent backup exists.
  • Schedule a short maintenance window. The command stops the application while it moves the data.
  • Confirm that the MongoDB 8.0 data and /secureone/data are on the same filesystem. On the same filesystem, the move is a rename. On different filesystems, the move becomes a full copy, the downtime grows with the size of the data, and retire refuses to run unless you add --allow-copy. The copy also needs free space equal to the data size.

The command refuses to run, and changes nothing, in the following cases:

  • The write-path services aren't all on MongoDB 8.0, or the stale-reference scan finds a reference to MongoDB 4.0. Finish the cutover first.
  • /secureone/s1.env isn't writable.
  • On a single node, <stack>_db8 isn't running.
  • On a three-node cluster, the three secureone8 members aren't all healthy.
  • The stack file pins the MongoDB image to a fixed tag, or the redeployed image isn't available on the host.
  • On a three-node cluster, the stack file hardcodes the MongoDB version or the replica set name. The stack file must read MONGO_VERSION and MONGO_REPLICA_SET instead.

Preview the changes​

Run the command with --dry-run to report every check and every action without changing anything.

For a single-node deployment:

s1 mongo-upgrade retire --dry-run

For a three-node cluster:

s1 --multi-node mongo-upgrade retire --dry-run

The output lists a retire plan. Review it, and resolve every FAIL message before you continue.

Run retire​

  1. Run the command from the node you used for the migration.

    For a single-node deployment:

    s1 mongo-upgrade retire

    For a three-node cluster:

    s1 --multi-node mongo-upgrade retire
  2. Review the plan, and answer the confirmation prompt. The prompt warns that the application stops for a few minutes and that after this step you can't return to the previous service naming.

For a custom stack name, place --stack <stack-name> before mongo-upgrade.

The command accepts the following options:

OptionEffect
--dry-runReports every check and action, and changes nothing.
--yes, --non-interactiveSkips the confirmation prompt. All checks still run.
--allow-copyProceeds when the data and /secureone/data are on different filesystems, which turns the move into a full copy.

Before it changes /secureone/s1.env, the command saves a backup copy named /secureone/s1.env.pre-retire.<timestamp>.

On a single node, the command performs the following actions:

  1. Pins MONGO_VERSION in /secureone/s1.env.
  2. Removes the settings that cutover added, so the stack defaults apply again.
  3. Stops the write-path services, then stops <stack>_db and <stack>_db8.
  4. Renames /secureone/data/db to the .mongo4.<timestamp> archive.
  5. Renames the MongoDB 8.0 data to /secureone/data/db, or copies it with --allow-copy.
  6. Redeploys the stack, with <stack>_db running MongoDB 8.0 on the canonical path.
  7. Compares collection counts before and after the move, removes <stack>_db8, and restores the write-path services.

The write path stays stopped through the redeployment, so nothing writes to the database before the comparison passes.

Exit codes​

Exit codeMeaning
0Retire completed.
1The command refused to run and changed nothing, or it failed early enough to roll back automatically. The command restores the write path against the previous services.
3The command failed after the redeployment, and the tool can't undo that automatically. The report names the exact recovery commands.

Recover from a failed retire​

  1. Read the report. When the exit code is 3, run only the recovery commands that the report names.
  2. Don't scale services back up or remove services manually unless the report tells you to.
  3. After you fix the cause, run retire again. The command picks up an interrupted run where it stopped, when it finds the archived MongoDB 4.0 data and a database service that serves the moved data.

If you can't resolve the failure, preserve the log and contact Netwrix Support. The log is in /secureone/logs/mongo-upgrade/.

Reclaim the disk space​

The tool keeps the MongoDB 4.0 data in the archive. Until you remove it, the archive is the only thing that keeps a rollback possible, and it uses disk space. When you're satisfied with the retired deployment, for example after a full backup cycle and a business day of normal use, remove the archive.

At the end of a successful run, retire prints the exact commands for each node: a sudo rm -rf <archive-path> command, docker volume rm <volume> commands, and a command for the checkpoint volume on the node where you ran run. On a cluster, the checkpoint volume exists only on that node. The tool never runs these commands for you. Run the docker commands with sudo.

warning

These commands are irreversible. After you run them, you can't roll back to MongoDB 4.0. Confirm that you have a verified, independent backup first.

Next steps​

See Verify and maintain the deployment after cutover for the checks to run after retire.