wiiisdom logo colored

360Plus Backup parallelization optimization

Introduction

The backup parallelization feature was implemented in the 2019.3 release. It already offers a significant boost in backup performances out of the box for customers that enable the option, but there are additional settings you can tweak to get the most of this feature.

Backup algorithm

There are three main components that are important to know to be able to understand the impact of the available parallelization options in the overall backup process:
  1. Resolvers: those services are in charge of running queries on the BOBJ database to retrieve the objects that should be considered by the task (based on the task selection) and if they should be backed-up or not (the delta behaviour, comparing the object's latest update time again the date of its latest 360Plus backup).
  2. Backuppers : those services are charge of creating a backup for objects that should be backed-up by the task. They'll create the BIAR file associated with the backup, add a new entry for the backup in the 360WebPlatform database, send the backup to the repository.
  3. Object stack: objects that are retrieved by Resolvers are sent in the Object stack, waiting to be processed by Backuppers.
Retrieving an object in the BOBJ database is faster than creating a BIAR of the object, and you'll always retrieve more objects that you'll actually backup in subsequent backup tasks (delta behaviour). For those reasons, there's by default (parallelization option disabled) more Resolvers than Backuppers: 4 Resolvers and 1 Backupper.

Number of objects to process in parallel (Backuppers)

This is the setting available in the Backup Tasks configuration form. This value represents the number of objects that will be backed-up at the same time, and it will be thus used to determine the amount of Backuppers running in a Backup Task.
The upper limit for this setting is the amount of logical CPU cores  on the server hosting the 360WebPlatform application. Going past this value won't affect the backup performances in a significant manner, and may even slow down the process compared to a lower value depending on the customer's environment.

Number of Resolvers

When the number of objects to process in parallel is set to its upper limit and customers still want to improve the backup performances, they can tweak the number of Resolvers (provided that they still have resources available on the server when the backup tasks is running).

The number of Resolvers in backup task is the greater of those two values:
  1. Number of objects to process in parallel setting
  2. Minimum number of Resolvers
This second parameter is 4 by default, but its value can be changed using the following Java Option:

-Dwp.backup.minresolver=X

There's less of an upper limit for Resolvers but you'll most likely end up reaching the BOBJ database bottleneck, where 360Plus sends too many requests at once for the BOBJ database.

You'd thus want to use this parameter to set a value higher than the value you're using for the Number of objects to process in parallel setting, especially when you're running delta tasks (large difference between the number of objects retrieved from the BOBJ database and the number of objects that will actually be backed-up).
You would also use this parameter to set up a lower value when your Backup Tasks are impacting your BOBJ database too much, as we're now running at least 4 BOBJ queries at the same time instead of a single one in the old algorithm.

Size of the Object stack

By default, the size of the Object stack is set to 1000.
If the stack is full, the Resolvers will have to wait for the Backuppers to remove objects from the stack before being able to retrieve new objects.

This value can be changed using the following Java Option:
-wp.backup.stacksize=X

The upper limit is directly correlated with the amount of memory available on the Tomcat server for the application.

This value should be lowered when the memory footprint of the Backup Tasks is too important for the available resources.
This value can be increased alongside the number of Resolvers, provided that there are enough resources available on the server.
Last updated almost 4 years ago