Migrate DataSource To Another Disk Failed!

Hi

We have DPM 2012 R2 installed with a 8TB iSCSI disk attached. Due to us running out of space we decided to add another 10TB iSCSI disk to the storage pool and use MigrateDatasourceDataFromDPM.ps1 to migrate all the data from the 8TB to the 10TB disk. Seemed straight forward but halfway through the migration DPM ran out of space on the 10TB disk, very odd as we are only copying 8TB to a 10TB disk.

My question is how do I find out what protection group data hasnt been moved and what is left to move? Basically what data is on what disk?

Below is the PowerShell I used and the error returned.

PS C:\Program Files\Microsoft System Center 2012\DPM\DPM\bin> $disk = Get-DPMDis

k -DPMServerName OurDPMServer

 

PS C:\Program Files\Microsoft System Center 2012\DPM\DPM\bin> ./MigrateDatasourc

eDataFromDPM.ps1 -DPMServerName OurDPMServer -Source $disk[0] -Destination $disk

[1]

 

Set-ProtectionGroup : The allocation of disk space for storage pool volumes

failed because there is not enough unallocated disk space in the storage pool.

(ID: 358)

Add more disks to the storage pool or reduce the volume size requirement.

At C:\Program Files\Microsoft System Center

2012\DPM\DPM\bin\MigrateDatasourceDataFromDPM.ps1:111 char:3

+         Set-ProtectionGroup $mpg

+         ~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: (:) [Set-DPMProtectionGroup], DlsE

   xception

    + FullyQualifiedErrorId : NotEnoughFreeSpace,Microsoft.Internal.Enterprise

   Storage.Dls.UI.Cmdlet.ProtectionCmdLets.CommitProtectionGroup

Set-ProtectionGroup : Non VSS Datasource Writer has recently been migrated.

You cannot migrate Non VSS Datasource Writer again until the recovery points

on the previous replica volume are available. (ID: 31224)

At C:\Program Files\Microsoft System Center

2012\DPM\DPM\bin\MigrateDatasourceDataFromDPM.ps1:111 char:3

+         Set-ProtectionGroup $mpg

+         ~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: (:) [Set-DPMProtectionGroup], DlsE

   xception

    + FullyQualifiedErrorId : ChainedMigrationError,Microsoft.Internal.Enterpr

   iseStorage.Dls.UI.Cmdlet.ProtectionCmdLets.CommitProtectionGroup

Remove-DPMDisk : Disk 1 cannot be removed from the storage pool because it

contains storage pool volumes. (ID: 355)

At C:\Program Files\Microsoft System Center

2012\DPM\DPM\bin\MigrateDatasourceDataFromDPM.ps1:113 char:2

+     Remove-DPMDisk $Source

+     ~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: (:) [Remove-DPMDisk], InvalidDiskE

   xception

    + FullyQualifiedErrorId : DiskContainsSpmVolumes,Microsoft.Internal.Enterp

   riseStorage.Dls.UI.Cmdlet.RemoveDPMDisk

PS C:\Program Files\Microsoft System Center 2012\DPM\DPM\bin>

February 13th, 2014 2:45pm

Hi,

Can you post the output of: Get-dpmdisk

When you select a disk in the DPM storage pool, it will list data sources that are located on that disk under the section "Protected data sources on this disk

Free Windows Admin Tool Kit Click here and download it now
February 13th, 2014 4:00pm

Hi Mike, thank you for your reply.

Sure the Get-DPMDisk out put is below

PS C:\Program Files\Microsoft System Center 2012\DPM\DPM\bin> Get-DPMDisk

Name          DiskStatus        NtDiskId TotalCapacit UnallocatedS DiskTypeLabe
                                               yLabel    paceLabel l
----          ----------        -------- ------------ ------------ ------------
QNAP iSCSI... Healthy                  1      8191 GB      2499 GB Dynamic
QNAP iSCSI... Healthy                  2     10751 GB      1686 GB Dynamic

yes under the "Protected data sources on this disk" section within the console i can see the remaining sources, but how would i list them using Powershell, if i use Get-DataSources it lists data on both disks.

Thanks

February 13th, 2014 6:17pm

Hi,

Save the following as MigrateSpaceNeeded.ps1 - then run inside the DPM Powershell using the verbose switch.

1) Change the properties of the powershell window layout to be 160 characters for width.
2) Run MigrateSpaceNeeded.ps1 verbose
3) The output will detail what data sources is on each disk and the space needed to migrate it.

param ([string] $global:verbose)
  $Version      = 2.1
  $ScriptName   = "MigrateSpaceNeeded"

function GetMigSizeByDisk
{ 
	param($disk, $extents, $dss, $Vols ) 
	[int64]$size = 0 
	#For all data source members on current disk 
    $disk_distinct = $disk.pgmember | sort-object volumesetid -Unique  
	foreach ($m in $disk_distinct) 
	{ 
		#we use ds to take sizes from rather than selects and sum proper extents 
		$ds = $dss | ? {$m.DatasourceId -eq $_.id}
        if ($verbose.toupper() -eq 'VERBOSE')
        {
    		write-host; write-host $ds -foreground red 
            $ds.ProtectableObjectLoadPath.Keys | out-file $outfile -Encoding ascii -Append

        }
        $RecoVolume = @($Vols | ? { $_.volumesetid -eq $m.VolumeSetId.Guid -and  $_.usage -eq 2 })        
        $RecoExpr = "Guidname='{0}'" -f $RecoVolume[0].GuidName 

        $RecoExtents = @($extents | ? { $_.guidname -eq $RecoVolume[0].guidname })
		# At least 1 recovery point volume extent lives on same disk as replica 
		# We must now add shadowcopy size to total 
		$size += $ds.ReplicaSize + $ds.ShadowCopyAreaSize #$includes all extents whereever they are   
        $DatasourceVolumeSize = $ds.ReplicaSize + $ds.ShadowCopyAreaSize #$includes all extents whereever they are   
        $replica = @($Vols | ? { $_.volumesetid -eq $m.VolumeSetId.Guid -and  $_.usage -eq 1 })
		$expr = "guidname= '{0}'"  -f $replica[0].guidname
        $replica2 = $extents | ? { $_.guidname -eq $replica[0].Guidname }
        if ($verbose.toupper() -eq 'VERBOSE')
        {
		  foreach ($replica1 in $replica2)
		  {
		      write-host "   Replica volume Size  - $('{0,8:N2}'-f($replica1.extentsize/$GB)) GB - Replica on Disk $('{0,3}'-f($replica1.ntdisknumber)) - Volume GUID " $replica1.Guidname -foreground green 
              "   Replica volume Size  - $('{0,8:N2}'-f($replica1.extentsize/$GB)) GB - Replica on Disk $('{0,3}'-f($replica1.ntdisknumber)) - Volume GUID " + $replica1.Guidname | out-file $outfile -Encoding ascii -Append

		  }
        }
		$expr = "VolumeSetId='{0}' and usage = 2" -f $m.VolumeSetID, $vols.usage
        $Recovery = @($Vols | ? { $_.volumesetid -eq $m.VolumeSetId.Guid -and  $_.usage -eq 2 })
        $recovery2 = $extents | ? { $_.guidname -eq $recovery[0].Guidname }
        if ($verbose.toupper() -eq 'VERBOSE')
        {
		  foreach ($recovery1 in $recovery2)
		  {
		      write-host "   Recovery volume size - $('{0,8:N2}' -f($recovery1.Extentsize/$GB)) GB - Recovey on Disk $('{0,3}' -f($recovery1.ntdisknumber)) - Volume GUID " $recovery1.Guidname -foreground green
              "   Recovery volume size - $('{0,8:N2}' -f($recovery1.Extentsize/$GB)) GB - Recovey on Disk $('{0,3}' -f($recovery1.ntdisknumber)) - Volume GUID " + $recovery1.Guidname | out-file $outfile -Encoding ascii -Append
		  }
          write-host "   Total Volumes size   - $('{0,8:N2}' -f($DatasourceVolumeSize/$GB)) GB"
          "   Total Volumes size   - $('{0,8:N2}' -f($DatasourceVolumeSize/$GB)) GB" | out-file $outfile -Encoding ascii -Append
          " " | out-file $outfile -Encoding ascii -Append
        }        
	}    
    
	return $size
} 


$ErrorActionPreference = "silentlycontinue"
Add-PSSnapin -Name Microsoft.DataProtectionManager.PowerShell
add-pssnapin sqlservercmdletsnapin100 -ErrorAction SilentlyContinue
$ConfirmPreference = 'None'

$global:MB = 1024 * 1024 
$global:GB = $MB * 1024 
$global:dpmservername = @(hostname) 
$ConfirmPreference = 'None'
$outfile = $ScriptName + ".txt"
out-file $outfile -Encoding ascii

#get sqlserver and instance
$DPMServerConnection = (Connect-DPMServer (&hostname))
$DPMServername = $DPMServerConnection.name
$DB = $dpmserverconnection.dpmdatabaselogicalpath.substring($dpmserverconnection.DPMDatabaseLogicalPath.LastIndexOf('\') + 1,$dpmserverconnection.DPMDatabaseLogicalPath.Length - $dpmserverconnection.DPMDatabaseLogicalPath.LastIndexOf('\') -1 )
$servername   = $dpmserverconnection.dpmdatabaselogicalpath.substring(0,$dpmserverconnection.DPMDatabaseLogicalPath.LastIndexOf('\'))

$VolTable = Invoke-Sqlcmd -ServerInstance $servername -Database $DB -Query 'select *  from tbl_SPM_Volume where VolumeSetID is not null'

$extentlist = Invoke-Sqlcmd -ServerInstance $servername -Database $db -Query 'select extent.diskid, extent.DiskOffset, extent.GuidName, extent.ExtentSize,disk.NTDiskNumber  from tbl_SPM_Extent extent join tbl_SPM_Disk disk on extent.DiskID = disk.DiskID'


$disks = @(Get-DPMDisk $dpmservername  | ? { $_.IsInStoragePool -eq $true}  |  sort -Property NtDiskId )
$datasourcelist = Get-Datasource $dpmservername

#Now go to work
cls 
write-host "$ScriptName Version $version`nFor datasource details, run with verbose switch`nExample: <script_name> verbose"  -f white

Write-Host ("`nEvaluating {0} DPM disk(s)...`n" -f $disks.count) -ForegroundColor yellow 
foreach ($d in $disks) 
{ 
	$MigSize = ( (GetMigSizeByDisk $d $extentlist $datasourcelist $Voltable)/ $GB).Tostring("N2") 
	Write-Host (" ==> to migrate Disk-{0} you will need $MigSize GB on the destination" -f $d.NtDiskID) -ForegroundColor cyan 
    " ==> to migrate Disk-{0} you will need $MigSize GB on the destination`n" -f $d.NtDiskID | out-file $outfile -Encoding ascii -Append
    " "                                                                      | out-file $outfile -Encoding ascii -Append
} 

# Based on LDMAndShrinkStats version = "1.6" 
$ExtentCount = Invoke-Sqlcmd -ServerInstance $servername -Database $DB -Query 'select count(*) as  TotalNumberOfDynamicExtents from tbl_SPM_Extent extent where extent.DiskID in (select DiskID from tbl_SPM_Disk where DiskType = 1)' 
$DSCount     = Invoke-Sqlcmd -ServerInstance $servername -Database $DB -Query 'select COUNT(*) as TotalNumberOfDatasources from tbl_IM_Datasource where ProtectedGroupId is not null' 
$volcount    = Invoke-Sqlcmd -ServerInstance $servername -Database $DB -Query 'select COUNT(*) as NumberOfVolumes from tbl_SPM_Volume where VolumeSetID is not null'

$dpmRemaining = [math]::truncate(600 - $volcount.numberofvolumes) 
$diskcount = $disks.count 

Write-Host "Total disks in DPM Storage Pool".PadRight(31)  ":" $diskcount
Write-Host "Total volumes".PadRight(31) ":" $volcount.NumberOfVolumes
Write-Host "Total extents".PadRight(31) ":" $ExtentCount.TotalNumberOfDynamicExtents
Write-Host "Total data sources".PadRight(31) ":" $DSCount.TotalNumberOfDatasources
$usedslots = 1 + $diskcount + (2*$volcount) + $ExtentCount 
$slotsRemaining = 2960-$usedslots 
$volRemaining = [math]::Truncate($slotsRemaining/3) 
if ($dpmRemaining -lt $volRemaining) {$dsRemaining = [math]::Truncate($dpmRemaining/2)} 
else {$dsRemaining = [math]::Truncate($volRemaining/2)} 
Write-Host "Number of non-colocated data sources that can still be added:" $dsRemaining
Write-Host "`nData source extent list..." 

"Total disks in DPM Storage Pool".PadRight(31) +  ": " + $diskcount            | out-file $outfile -Encoding ascii -Append
"Total volumes".PadRight(31) + ": " + $volcount.NumberOfVolumes                | out-file $outfile -Encoding ascii -Append
"Total extents".PadRight(31) + ": " + $ExtentCount.TotalNumberOfDynamicExtents | out-file $outfile -Encoding ascii -Append
"Total data sources".PadRight(31) + ": " + $DSCount.TotalNumberOfDatasources   | out-file $outfile -Encoding ascii -Append
"Number of non-colocated data sources that can still be added: "+ $dsRemaining | out-file $outfile -Encoding ascii -Append
" "                                                                            | out-file $outfile -Encoding ascii -Append
"Data source extent list..."                                                   | out-file $outfile -Encoding ascii -Append
" "                                                                            | out-file $outfile -Encoding ascii -Append

$DSwithExtentsTable = Invoke-Sqlcmd -ServerInstance $servername -Database $DB -Query 'SELECT (select DataSourceName from tbl_IM_Datasource ds2 where DataSourceId = DS.DataSourceId) as DatasourceName, 
(select pg.FriendlyName from tbl_IM_Datasource ds2 join tbl_IM_ProtectedGroup pg on ds2.ProtectedGroupId = pg.ProtectedGroupId 
where DataSourceId = DS.DataSourceId) as ProtectionGroupName, 
COUNT(Extent.DiskID) as NumberOfExtents, replica.PhysicalReplicaId as PhysicalReplicaId, DS.DatasourceId as DatasourceId 
FROM tbl_SPM_Extent Extent WITH (NOLOCK)  
    JOIN dbo.tbl_SPM_Volume Volume WITH (NOLOCK)  
        ON Extent.GuidName = Volume.GuidName  
    JOIN dbo.tbl_PRM_LogicalReplica Replica WITH (NOLOCK)  
        ON Replica.PhysicalReplicaId = Volume.VolumeSetID  
    JOIN dbo.tbl_IM_Datasource DS WITH (NOLOCK)  
        ON Replica.DataSourceId = DS.DataSourceId 
GROUP BY DS.DataSourceId, Replica.PhysicalReplicaId 
Order by NumberOfExtents desc'
$DSwithExtentsRows =   $DSwithExtentsTable
$physicalReplicasPrinted = @{} 
foreach ($row in $DSwithExtentsRows) 
{  
    if ($physicalReplicasPrinted.Contains($row["PhysicalReplicaId"])) {  
        $row.Delete()  
    } else {  
        $physicalReplicasPrinted[$row["PhysicalReplicaId"]] = 1;  
    } 
} 
$DSwithExtentsTable.AcceptChanges() 
$DSwithExtentsRows | ft AutoSize 
$DSwithExtentsRows | ft -AutoSize | out-file $outfile -Encoding ascii -Append
Write-Host "`nReplica colocation counts..." 
$PhysReplCount = Invoke-Sqlcmd -ServerInstance $servername -Database $DB -Query 'select physicalreplicaid as PhysicalReplicaId, COUNT(*) as Count from tbl_PRM_LogicalReplica where physicalreplicaid is not null and datasourceid is not null group by PhysicalReplicaId'
$PhysReplCount | ft AutoSize 
$PhysReplCount | ft AutoSize | out-file $outfile -Encoding ascii -Append
Write-Host "`nLDM alerts list..." 
$LDMAlerts =   $db.ExecuteWithResults($LDMAlertsCmd).Tables[0].rows 
if ($LDMAlerts.Count -gt 0 ) {  
    $LDMAlerts | ft AutoSize 
    $LDMAlerts | ft -AutoSize | out-file $outfile -Encoding ascii -Append
} 
else {write-host "None found!" -f white} 
Write-Host "`nDone!" 
#--------------------------------- 

 notepad $outfile

Free Windows Admin Tool Kit Click here and download it now
February 13th, 2014 6:43pm

Successfully migrated now, thank you for your help Mike.
February 17th, 2014 4:54am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics