my personal blog about systemcenter

All posts in Disaster Recovery

Protecting your secrets, one more step to remember

Categories: Active Directory, AD, Backup, Disaster Recovery, Password, TSM
Comments Off on Protecting your secrets, one more step to remember

If you are using hosted backup with TSM there is one more step to cover when people leave the org

The protection for many hosted backups are

Protection against “rouge” TSM Administrator

Client Side Encryption

Protection against “rouge” Backup Administrator

Node ID

Node Password (separation of duties one for password one for encryption)

And the last one is the issue here as its often not rotated, default TSM is 90 days but looking at different hosted TSM password is often set to no expire

This is not a TSM problem but a problem with password rotation

In the perfect world, the NodeID password and the encryption is not known by the same person, but then nodeid / password / secret is in registry so an AD admin can access this


TSM BA Client installed on


Starting the TSM client , prompting for Node Password on first backup


Ready for Action


Starting the first backup , prompts for encryption key , and after a short while the backup is completed


On a rouge server, outside of the environment we install the TSM BA Client and reuse the nodeID and password from the disgruntled backup admin


Adding the nodeid and nodepassword


And we restore a dummy file to see that’s its working, and is prompted for the encryption key

dsmc q b “{DEMODC01\SystemState\NULL\System State\SystemState}\ntds.dit” -sub=y


If we can’t remember where ntds.dit is located we can search for it

rest “{DEMODC01\SystemState\NULL\System State\SystemState}\\DEMODC01\C$|\WINDOWS\ntds\*” C:\EVILDC\ -sub=y


And we can restore the files


And we now have something we can attack , if we boot up in a winPE enviroment we can follow the procedure for system state and have a working domain controller


If the attacker had access to the domain controller aka disgruntled former employee the password and encryption is available on the source node in registry , since TSM used both the password and the encryption to access TSM server and backup/restore data it needs to be stored somewhere that the service can access

It’s very hard to protect anything from a domain admin even with the assume breach state of mind


So, we can logon without getting prompted for credentials/encryption

So what can we do

First off , prevent people from being disgruntled

And since we can’t control human nature change the password on the nodes, either scheduled or when high privilege staff leaves or both, and again the default for a TSM node is that it will be changed


Single Node example, log on the TSM , change password


Something old Something New


And Success , and password change can be scripted so cycling the password shouldn’t be a big issue


And our EvilDC can’t access TSM anymore and everything is back to normal

Deploying Data Protection Manager in a dedicated domain

Categories: Active Directory, Data Protection Manager, Disaster Recovery, DPM, Hyper-V
Comments Off on Deploying Data Protection Manager in a dedicated domain

Data Protection and the ability recover data is key to keeping your job and your company alive.

The demo setup thats is going to be used in this post are

  • PROTECTDC01 Domain Controller in the PROTECT Forest
  • PROTECTDC02 Domain Controller in the PROTECT Forest
  • PROTECTDDPM01 Data Protection Manager Server in the PROTECT Forest
  • FABRICDC01 Domain Controller in the FABRIC Forest
  • FABRICDC02 Domain Controller in the FABRIC Forest
  • FABRICHV01-04 Hyper-V HyperConverged Instal
  • FABRICHVC01 Hyper-V Cluster with member FABRICHV01-04
  • WORKLOAD01-05 Virtual Workload in the FABRIC Hyper-V Cluster

As this is a test enviroment everything are stuck on one box.

For the real world deployment the FABRIC and PROTECT domain must be seperated , the whole point in this post will be if you for some reason get compromised in your FABRIC domain , you will still have access to the PROTECT domain and maintain the ability to recover your data.

This also means that in a larger enviroment you can easier seperate the roles so one team wont have access to both source and target of backup data

We do in the example log in interative on the fabric domain , so if the host is compromised before agent install the protect domain is going down the same path , so there is still some work to be done but beats having everything in one domain.


On the PROTECT domain setup DNS forwarders to the FABRIC domain


And in Reverse to get name resolution up and running up between the two forests


Setting up the trust


Setting up the trust


for this test forest-wide is used , tighter security can be used with selective authentication


On the 4 Hyper-V Hosts we add the DPM account from the protect domain


We then add the DPM agent to all Hyper-V hosts and run the

SetDPMServer –dpmservername  , this connects the Hyper-V host to the remote DPM server


On the data protection manager , we use Attach Agents


And we add the 4 Hyper-V hosts manually


And we now have all 4 servers


use credentials in the fabric domain or the dpm account to attach the agent




Create a protection group browse to the VM’s and add them

And we can now backup from a dedicated domain from the Fabric domain

DPM Virtual Machine Recovery Hyper-V Clustering

Categories: Broken, Data Protection Manager, Disaster Recovery, DPM, Hyper-V, Virtual Machine Manager, VMM
Comments Off on DPM Virtual Machine Recovery Hyper-V Clustering

One task needed for planning for disaster recovery with DPM is the naming of the VM from a clustering point of view.

If all VM’s are created through VMM then will be prefixed with SCVMM but in this example we have created one VM from Cluster Manager and one from VMM

The scenario for this type of recovery is a “dead” cluster or VM’s that was deleted by mistake.

Being able to recovery directly would be preferred but thats not currently a option


This leaves us with VMMTESTB prefixed from a naming point of view with SCMMM and one without when selecting the VM’s from DPM


And the view from Cluster Manager


And after creating the protection group we are ready to test a recovery


And our view from VMM


If we need to recover the VM to the orignal source we can just select recover


And we can recover to the original instance


And to provoke a error


We kill the VM from both cluster and hyper-v


If we then tries to recover a VM to its Original Instance


The recovery dies as we dont have the origianal VM


So we need to create a empty VM


Point it to a dummy location ,the name needs to match the original VM name


And no VHD attached


In VMM we can not see the new VMTESTA and the VMTESTA that we delted , this scenario is if the cluster dies and needs to be rebuild but VMM isnt “gone”


We can then restore the VM directly


We then have the temp clustered VM , we need to delete that


And also on the Hyper-V Host , this can also be done from VMM to avoid having to open the settings to find the restore VM


And Configure Role


And select Virtual Machine


And Enable it for HA


And last clean up if the VM wasnt deleted from VMM , remove the VM that have missing state


Had a question today about the ability to recover the SQL master database if the SQL server wasnt operational.

To set the scene for the converation we did the following


Installed the Data Protection Manager on the SQL 2012 server , created a brand new fresh and shiny protection group , added the SQL instance


Created short term recovery this was after all just a test


And a backup every hour kept for the last 5 days



And as always with Data Protection Manager Sucess is an option 🙂


And a few minutes later we now have a full backup of the SQL server


And lets try to add the human factor




And after that we have a non functional SQL Server



And as we can see the reason if lack of master databases



DPM to the rescue , enter recovery select the master database


Option One , Recover to original instance (otherwise restore to folder and copy the files manually)



And leave database operational as we are not recovering from additional logs



Verify server/files


And 35 seconds later the database is recovered without incident



Recovery is complete , SQL instance is restarted and we can fire the blogger that deleted the master database


Marcel van der Berg posted about the new Hyper-V recovery Manager


Read his brilliant post and then

If you would like to be considered for this program please complete the Microsoft survey located here. Thank you in advance for your responses! We will only be contacting those of you that have been accepted to participate.

Go Click NOW Smiley