FlashBlade: configure S3 as a Veeam Backup & Replication repository - Scale-Out/SOBR Capacity Tier
[ NOTE: machine translation with the help of DeepL translator without additional proofreading and spell checking ]
It's been almost a year, but you won't find much information about FlashBlade S3 in conjunction with Veeam Backup & Replication as object storage.
Veeam's HCL "Veeam Ready database" already lists several Pure Storage systems in a wide range of categories:
FlashArray with //X, //M and //FA-420 with their Storage Snapshot integration.
FlashBlade as backup repository and "Object Ready" object storage/S3. The latter is what this article is about.
FlashBlade as a "Veeam Ready Object" is supported since Veeam B&R 9.5.4.x and is listed in the HCL (Veeam Knowledge Base > 2944) as tested, validated and classified object storage for Cloud Tiers since May 2019.
The Purity tested at the time corresponded to 2.2.11. and I could not find any issues in my test environment with a FlashBlade Purity 3.0 during my tests, so I assume compatibility from Purity 2.2.11 and newer!
General
An object store in B&R is part of the "Capacity Tier" and extends scale-out repositories with the ability to offload existing backups from the "Performance Tier" directly to cloud-based object stores such as Amazon S3, Microsoft Azure Blog Storage, IBM Cloud Object Storage and other on-prem S3 compatible systems. In principle, an on-prem S3 compatible system is any system that complies with the Amazon S3 protocol (Veeam support should be mandatory in production environments).
For FlashBlade we talk about "other on-prem S3 compatible systems/S3 Compatible Object Storage".
In addition, object storage can also be used as a repository for NAS backups.
Scale-Out Backup Repository
A Scale-Out Backup Repository (SOBR) is a logical unit that groups multiple backup repositories (DAS, NFS, SMB, S3...) - so-called extents. So when you configure a SOBR, you create a storage pool of multiple storage systems to bundle their capacity. Within the SOBR you can then create a performance pool aka "Performance Tier" for maximum performance and high availability storage and also a capacity pool aka "Capacity Tier". For long-term retention, one can instruct B&R to move data from performance tiers to capacity tiers, saving valuable storage resources of the more expensive performance tier.
In addition, a SOBR can scale out at will, so you have the choice of increasing your existing repositories (scale-in) or adding a new system (scale-out). This eliminates the need to move already stored backup data to a new storage system.
Extents
A SOBR can contain one or more extents. An extent added as a backup repository can no longer be used as a regular repository in other jobs. Jobs that are to use the extent must therefore now be adapted to the SOBR.
The policy of the distribution of backup data between the individual Extends can be set in the SOBR, here you can choose between two policies:
Data locality
Performance
"Data locality" places backup data of a backup chain on the same SOBR extent. This ensures that backup chains are always extent-independent and complete.
Performance", on the other hand, distributes backup data/chains across extents. One can define separate extents for keeping full/incremental backups. With the performance policy, one can achieve improved performance, as well as minimize/distribute the I/O load on the backup storages. Sufficient bandwidth when connecting the storages to the proxy server(s) is a must here.
Configuration
FlashBlade
Preparing the S3 bucket on the FlashBlade is done quickly.
We switch to the Storage > Object Store tab and create a new account "veeam-s3" (you can also use existing accounts - but this is not recommended, because then other already existing users have access to the Veeam bucket). Within the account, the buckets and users are then created by us, so you can also see it as a logical unit at this point.
Immediately after that I create a user "user1" in the "veeam-s3" account. For now without generating an access key, we will do this at the end. Only the bucket is missing, this is also done in no time - + -> Create Bucket "bucket1".
As mentioned above, we still need an Access and Access Secret Key to set up the object storage in Veeam. The secret access key must be saved, as it can no longer be viewed - if it is lost, it must be regenerated.
It is also possible to export the corresponding credentials via JSON or CSV for documentation purposes and save them.
Veeam
1. Setup repository
The integration of the S3 repository is also unspectacular. The wizard is started in the VBR console via Backup Infrastructure > Add Repository > Object storage > S3 Compatible.
We assign an appropriate repository name and switch to the "Account" tab with Next. Here we specify the service point - this is the intended S3 data interface (or the "data interface") on the FlashBlade. The IP address is sufficient here, no protocol type or S3 URL needs to be specified.
As region we enter on-prem (mandatory on-prem, otherwise the wizzard fails).
The previously created credentials for the object account including access and secret key are also stored here. A description (optional) should simplify a later assignment in the production environment.
It may now be that another certificate message appears: since this is a test environment, I continue with the setup without paying attention. In production environments one should import the certificates accordingly from the FlashBlade (Settings > System) on the Veeam servers.
Veeam recognizes the created buckets for the deposited user (Register Account) and you can select the corresponding bucket.
Finally we have to create a folder (start directory) in the bucket for the repository. I named this "veeam-s3-data". Our backups are then stored in the created folder.
Optionally, we would still be able to make advanced settings:
"Limit object storage consumption to": set up a limit for the S3 repository. Depending on the definition, the maximum defined capacity is available. After exceeding the limit, no new tasks will be written to this repository - started tasks will be completed.
"Make recent backups immutable for": here you can prohibit the deletion of data from the object store for X days. Here VBR uses the "S3 Object Lock" feature, which the underlying object store must support.
The backup object store is now set up as a repository.
2. Setup Scale-Out Repository
The target architecture or backup strategy should be as follows: the customer's backups (VMware backups as an example) are stored daily in the DR data center on a ReFS volume (RDM of the volumes) at the backup proxies. The ReFS volumes are provided here by a dedicated FlashArray for DR purposes. Backup data that has been residing in the SOBR performance tier for 14 days is to be offloaded to the capacity tier/FlashBlade.
The plan/strategy (a full-fledged "FastRestore" concept) can be depicted as follows:
The corresponding configuration of the scale-out repository is done as follows:
I assign a name "SOBR1" for the scale-out repository and add a short description.
In the performance tier we select our primary backuptarget with "Add" - in the example the FlashArray with its ReFS volume at the backupproxy (RDM).
In the advanced settings, properties of the extent can be defined for the performance tier. These are self-explanatory and we leave them at the default values.
Now we need to define a placement policy. You can read about the differences between the two policies in the "Scale-Out Backup Repository" section of the article above.
In the test environment, we have only one extent in the tier and setting it as a performance policy would therefore not make sense at this point. In the advanced settings, the above-mentioned settings for distribution by backup data type (Full/Increment) can be set as required.
Lastly, we need to add the extent for the Capacity Tier - our flashblade object storage - and set the swapping of backup data after 14 days. There are more configuration options available:
Definition of a time window for the relocation. Usually not interesting for us with on-prem S3 if there is sufficient internal bandwidth.
"Copy backups to object storage as soon as they are created": would always initiate an immediate transfer (copy) of the backup data from the Performance Tier to the Capacity Tier.
"Move backups to object storage as they age out of the operational restore window" > Override: gives the possibility to swap backup data out of the performance tier prematurely if the defined threshold of used memory would be reached (prevents overfilling of the performance tier).
"Encrypt data uploaded to object storage": Set up data encryption when transferring to S3 and software level/Veeam (doesn't make sense with on-prem S3 and FlashBlade as data is stored encrypted at hardware level - Purity takes care of this).
The configuration of the Scale-Out Backup Repository is now complete and you can use it to configure your backup jobs.
Conclusion
With the Capacity Tier Option of Backup & Replication, Veeam offers a simple and straightforward way to move backup data to S3 storage (without major configurations). With the validated Veeam S3 compatibility in FlashBlade, it is also possible to move to high-performance S3 storage from Pure Storage. A way to also consume the advantages of S3 and the features of FlashBlade (including single namespace, compression) and not have to compromise on performance.
More info - Links
All officially published setting options in the GUI but also CLI can be read via the "on-board" user guides of the Pure Storage systems.
Click on "Help" in the Purity main menu.
The User Guide is structured like the main menu and can be opened downwards. A search function is also integrated - within here you can also search for keywords.
WEB: Pure Storage (Pure1) support portal - Ticket system and support *(requires registered FlashSystems)
PHONE: Pure Storage phone support: GER - (+49) (0)800 7239467; INTERNATIONAL - (+1) 650 7294088
WEB: Pure Storage OFFICIAL blog