Home- My Twitter Stream
- My LinkedIn
- My Blog
- My Last.fm Music
- Goodreads
- 8tracks
-
Tweets
- You'll Never Want to Skydive After Watching This [Video] http://t.co/4wpb4h52
- @JoeBaguley @SteveBB @theRealWakey files are portable and interoperable. We'll need files until we can move data seamlessly between clouds.
- @JoeBaguley @SteveBB @theRealWakey It's the marketing hype which drives me nuts...a tablet is currently a big phone. Win8 *may* change that.
- And no, a tablet is not a PC. There is overlap and your use case may allow you to use a tablet more than a PC, but they are not the same.
- Really drives me crazy how people confuse consumerisation with a surge in mobile (esp. tablet) devices. They really are not the same thing.
- Ooh Roger Black talking at the SCC Sales Kickoff! My hero from my 400m running days :-)
- "@SteveBB: 8 Bullets point for SMTC and only 2 for TaaS - in your face ! @DavidWarburton ;)" lol :-D they just don't realise its true power
- RT @DuncanYB: Want a free Kindle version of the VMware vSphere 4.1 HA and DRS technical deepdive? Pick it up now! http://t.co/u59Fgn6I
- LOL RT @seankalahar: RT @sintaxasn Oh SQL, you cockjuggling thundercunt. Why do you mock me so? < THE tweet of the day
- http://t.co/iQ3muz0c hands down the best noodles I have *ever* had. Simply amazing. You owe it to yourself to order some 'pad thai chicken'!
- Silicon trick for next-gen memory http://t.co/OO2gydv6 / what VDI IOps problem?
- My ex-Windows Phone locks up 2nd time in a row. Hard Reset again. Still loving my One X but do miss some WP7 features, keyboard the most!
- Chameleon: The New Kickstarter Project That Could Revolutionize Tablet Homescreens http://t.co/E9iKPoRM
- "@fitzpas: I can't breathe... The 5-year-old has designed a Bat Man Land rollercoaster... http://t.co/DlA7xRLJ" lol that is pure awesome :-)
- UK based TVCatchup for Android released in beta http://t.co/BC3bJzmR
- "@thurrott: Is Time Running Out for Windows Phone? http://t.co/yxRCJGcH" I hate to agree but have to. Too much uncertainty for current users
- @kirkdavis I've found battery okay. Way better standby time than my WP HD7. You got the battery fix yet?
-
Info
- Solutions Architect at SCCInformation Technology and Services | Birmingham, United Kingdom, GBSummaryExtremely motivated and dedicated IT Professional with experience in many systems. Diverse career with roots as a developer, through hardware and software support to technical leads.
Extensive knowledge of Microsoft SharePoint (WSS and MOSS), Netapp SANs, VMware ESX, TCP/IP networking, Powershell and VBscript scripting. Works well under pressure and finds it easy to relate and communicate to senior managers and colleagues alike.
Fluent in French. Currently studying towards VCP.Specialties: Wintel Infrastructure: - up to Windows Server 2008 R2 (AD, AGPM and Group Policy Preferences, DNS, DHCP) - Terminal Server farms and Windows Load Balancing - HP blade enclosures and servers - VMware ESX and ESXi Storage & Backup - Netapp filers (CIFS, iSCSI, Fibre Channel networking, LUNs, NFS) - Symantec Netbackup 6.0 (including scripting of customised reports) - Tapeless disk-to-disk backups Scripting & Development - Strong Powershell and VBscripting (WSH, ADMI, WMI)Experience- Aug 2011 - PresentSolutions Architect, Data Centre Services / SCC
- Mar 2010 - Aug 2011Design and Implementation Specialist / SCCDesigned and built all major components (hosts, storage, SAN, network) of our new VMware vSphere 4.1 environment from the ground up. Migrated existing ESX 3.5 VM's to new infrastructure. Implemented SAN level backups of VM's to ease network congestion and allow flexible backup testing. Next job is to implement Site Recovery Manager... Also currently doing a lot of work around SAN (EVA 8400, and 8100) performance optimisation for SQL 2008, Exchange and Oracle. Setting up inter-site SAN replication (using HP EVA Continuous Access over FCoE). Learning loads around SAN and SQL best practises...turning in to a bit of a storage geek.
- Jan 2007 - Apr 2010Systems Engineer / Robert Half InternationalThis role began as 2nd line technical support for local and remote desktop users and also systems administration of over 300 servers in the UK and Belgium which served our 2000 employees across EMEA. Supported systems included Windows 2003 Terminal Services farms, Exchange clusters, Blackberry Enterprise Servers and Symantec Enterprise Vault. My scripting skills and diverse experience meant that I progressed through the company to perform more senior/3rd line support, mentored junior members of the team and acted as technical lead on a number of projects - the most recent of which was a deployment of a disk-to-disk backup and DR solution which made use and expanded upon our existing Netapp infrastructure. I was also responsible for creating and maintaining login scripts and created numerous automated scripts and processes to maintain and report on server health, provide secure functionality to the helpdesk with their limited user accounts, and report on backup status (from Symantec Netbackup - this script alone saved the business 1-3 man hours a day).
- Jun 2005 - Jan 2007IT Manager / WAA LtdResponsible for all IT support, management and infrastructure.
- Sept 2002 - Jun 2005IT Manager / Parkway GraphicsImplementing new technologies (Windows Server 2003 AD, Exchange 2003), installing physical network infrastructure, website coding and design, OS 9 and OS X support
- Jan 1999 - Sept 2002Web developer / AonCame on board as a junior web developer to aid with the widely used intranet. Soon started work on client pension sites providing a lot of the back end ASP code. Hugely developed my JavaScript skills providing some unique approaches to form validation and submission. Started building and configuring development servers based on Windows Server 2000 and IIS 5.
- Jan 1999 - Sept 1999Web Developer / Safix Lt
EducationAdditional Information-
Websites:
-
Interests:Martial Arts, acoustic guitar
-
Posts
- Mar 09, 11
So I’ve noticed a rather worrying feature of ESXi (I’m on 4.1 U1) in the configuration that I have. I’ve created our ESXi servers to use a vDS for VM traffic but for management, vMotion and FT I am sticking with the local vSwitch0. vSwitch0 has three active NICs and three port-groups, one for each of the above VMkernel tasks (management, vMotion and FT).
These port-groups have the NICs set to a specific order with only 1 active and 2 in standby ensuring that, although only 1 NIC is actively used for each portgroup, the two other NICs are there for failover. A few pics may explain this better….
vSwitch0 Overview
vSwitch0 vSwitch NIC configuration (click to enlarge)
Management port-group configuration
Management NIC configuration (click to enlarge)
It’s also worth noting here that, since we have Enterprise Licenses, I decided to make use of Host Profiles to help me set up other ESXi hosts. So I configured one ESXi server the way I wanted it and used it to create a baseline host profile. This profile was then applied to all remaining hosts. Nice, quick deployment of a standard config. Good times!
Server Down!
My problems started when I wanted to change the IP address or VLAN of the Management port-group. Actually problems arose when I made a change to ANY port group in vSwitch0. My most recent problems were when I wanted to change the VLAN that the vMotion port-group used. So I changed the VLAN and associated IP address for the vMotion PG and suddenly my host went offline!
What seemed to happen was that the new IP and VLAN I applied to the vMotion PG was actually applied to the Management PG!!! Seriously not good. Especially when your cluster has HA enabled which means that vCenter dutifully starts powering down VM’s and bringing them online to other hosts in your cluster! Queue lots of alerts and irate application owners.
After some digging around and finding some VMware Community posts I realised that the problem was due to an unconfirmed bug. It seems to be down to the way that Host Profiles are applied; specifically the order in which port-groups are created on the host.
If you look again at my first pic you’ll notice that the Management PG does not use the first vmkernel nic ‘vmk0′. This is actually assigned to the vMotion PG.
To me this is a massive problem and from my point of view, the “fix” is to make sure that the Management PG is using vmk0.
Reassigning vmk0
- The first thing you want to do is disable HA on your cluster. Otherwise when you make your change and temporarily loose connection to your host all your VM’s when vCenter detects that host has gone offline.
- Next, you’ll need either physical or ILO access to the DCUI. SSH won’t cut it as we’re going to loose network connection for a while. Once you’re on the machine hit ALT-F1 to go to the local TSM (assuming you have it enabled) and logon as root or admin user
- Now we want to remove all port-groups apart from the Management PG. We could do this via the VI Client, but lets do this at the command line being as we’ve made the effort to get here…List your vSwitch0 port-groups (and show vmknic’s):
Here, again, you can see that the Management port-group is using vmk3. This is bad, mmmkay? The other odd thing, and something I’m assuming is not good, is that the MAC shown for the Management PG is that of the physical NIC. This shouldn’t be the case. This should be a virtual MAC starting with the VMware specific range; 00:05 - The following commands removed my vMotion and Fault Tolerance port-groups. The affect of this is that their vmknics are destroyed:
esxcfg-vmknic -d vMotion esxcfg-vmknic -d 'Fault Tolerance'
Now the fun part. We run exactly the same command but on our Management PG. Yes. You will loose network connectivity to the host, but that’s why we’re using ILO, right?
esxcfg-vmknic -d Management
- After we’ve blown away our Management PG we can recreate it. When we do, it will automatically be given vmk0:
esxcfg-vmknic -a -i 10.32.202.11 -n 255.255.255.0 Management
You’ll also notice our new PG has a correctly assigned virtual MAC (starting 00:05). - At this point you’ll want to go back to the VI client and configure the new PG the way you want including adding additional pNIC’s to the vSwitch and setting failover order for the PG’s.
Now I’m able to make changes to my vMotion and FT port-groups without affecting my Management PG.
Let me know if you’ve been affected by this and how you’ve got around it.
My next step is to take another host-profile baseline and ensure that, when reapplying it to the other hosts, the vmknic assignments are consistent.
- Nov 03, 10
I’ve been plagued, for some time, by a really annoying problem in my home lab. Being as I run everything from the one VM host (ESXi) it is also home to my Windows 2008 domain controller.
Recently I had installed Veeam Backup and Replication to finally start backing up my VM’s (even though it’s my home/test environment I would be a broken man if I lost everything on there!). I installed Veeam on to the Windows 2008 DC – against all common sense of course, but RAM is low in my ML110 and I couldn’t afford to create another Windows VM.
I had configured Veeam to backup my guest’s using the vStorage API – Virtual Appliance mode. Backups of my other Windows guests ran well but I noticed some problems with the DC VM backup after a short while. It was around this time that I also noticed that the hard disks of this guest OS had automatically changed to Independant Nonpersistent! This would have been bad enough for any machine (as soon as the guest powers down you loose all changes), but for a DC – as you can imagine – it’s a nightmare!
This post on Experts Exchange confirmed – to some extent – that Veeam was the culprit and there wasn’t some other weird force at work: http://www.experts-exchange.com/Software/VMWare/Q_26310902.html
I disabled the DC backup via Veeam but the problem remained that my VM’s disks were Independant. The vSphere Infrastructure Client does not allow you to change disk modes on the fly so I was stuck – as soon as I shut my VM down to change the disk type I would loose all changes:
Thankfully, PowerCLI came to my rescue!
After connecting to my ESXi host using
connect-viserver <esxi_ipaddress>
All that was needed was a simple command:
Get-HardDisk -VM <vm_name> | Set-HardDisk -Persistence "Persistent"
Success!! At least partially. My primary (system) disk had successfully reverted to a persistent disk. However, my second disk had not and PowerCLI had actually errored :
CapacityKB Persistence Filename ---------- ----------- -------- 68157440 Persistent [localdisk01] vm_guest/disk1.vmdk Set-HardDisk : 04/11/2010 00:49:44 Set-HardDisk Another task is already in progress. At line:1 char:42
I haven’t yet figured out what task it is that is supposedly in progress, but I certainly can’t find one. It may be worth noting that my primary disk is on local SATA while the second disk is on an NFS share.
Now, at least, when I run Get-HardDisk, it shows that my system drive is Persistent:
- Oct 25, 10
This post has been sat in my WordPress Drafts folder for sometime since I no longer use local storage this way. I decided to post it however as (a) it’s a good learning curve for ESXi work and (b) others may have more luck that me.
I recently acquired three 1TB drives and decided to do something about my lack of storage at home. Always trying to make best use of existing kit (and save money) I decided to stick the drives in to my HP ML110 and try something in a VM instead of doing the sensible thing of lobbing them in to a dedicated NAS box.
After wasting a few hours I realised that the onboard SATA RAID controller of the ML110 just can’t do RAID5 and to make matters worse, when I gave up and created a RAID1 array with a hot spare, vSphere 4.1 didn’t recognise the array and instead saw the drives as 3 individual drives. I saw this as a chance to try out the WAFL-alike ZFS file system. FreeNAS had been my NAS of choice recently so I chose to try ZFS in that.
I point blank refused to create 3 1TB VMDK’s (one of each of the three drives) so I set about figuring out how to create Raw Device Mappings (RDMs) of the local SATA drives. There were a couple of posts on the net that got me a little closer, but no guide/article had the whole thing down, so that’s my aim with this blog post.
Step 1
Once you had your drives installed, SSH to your ESXi box (now even easier in vSphere 4.1) and go to the /dev/disks directory. There, if you perform a ls -l, you’ll see your drives listed:
Ignore the instances of your drives which show them as VM stores (vm1.*****). We want to look at the raw devices.
Step 2
Now move to the /vmfs/volumes folder. Here you can see your existing local datastore(s). If, like me, you had a solitary hard-drive, you’ll just see localdisk01 or whatever you chose to name the local datastore:
Step 3
Now we are going to use the vmkfstools utility to create our RDM’s. Remember that a RDM is just another VMDK, but instead of the VMDK pointing to a xxx-flat.vmdk file (which is the actual virtual hard disk), the VMDK points to our physical device. Being as we still need to create this VMDK file we need to save it somewhere. Since we just have the one local datastore, we are going to create the RDM VMDK files in it’s root.
The following command creates the RDM VMDK for us:
vmkfstools -z /vmfs/devices/disks/<name of RAW device from Step 1> <location to store VMDK>/<RDM name>.vmdk
In my personal example below, I am creating an RDM called rdm_WD2DWCAVU0477582.vmdk and it is being stored in the location /vmfs/volumes/localdisk01/ I chose the name of the VMDK to match the name of the serial number of the physical drive (and what is shown in Step 1) to help with troubleshooting in the future when I get an inevitable drive failure). You can call your RDM’s whatever you wish.
The name of the RAW device (t10.ATA____WDC_WD10EARS2D00Z5B1__________________________WD2DCAVU0477582 in my example) you will have noted from Step 1 when you listed all local devices attached to your ESXi host. This is why the tech Gods created Copy n Paste! You will want to copy the full device name as shown in Step 1 in to the vmkfstools command.
Step 4
Once you have repeated the steps for all of your local SATA drives, you can navigate to where you created the RDM’s (in my case /vmfs/volumes/localdisk01) and perform an ls -l *.vmdk to see the new VMDK’s you have created:
Don’t panic – the xxx-rdmp.vmdk files will reflect the size of the RAW devices they are mapping to, but rest assured it will be taking no more space than a few bytes on your local disk!
Step 5
You can now add your RDM’s to an existing VM. vSphere doesn’t recognise this as a true RDM (to a SAN) so you just browse the local disk datastore for the VMDK files that we created.
Edit the properties of an existing VM and click Add…
Step 6
Select Use an existing virtual disk and click Next >
Step 7
Click Browse. You now need to navigate your local datastore and select the VMDK’s that we created in Step 3).
Once complete you will be shown a confirmation window. Repeat Steps 5 through 7 to add additional RDM’s to your VM.
Step 8
You should now see your new Hard Disk’s in your VM and vSphere will correctly identify them as Mapped Raw LUN.
NOTE: One thing I forgot to show in the screen shots, is that you should create your RDM’s on a new SCSI controller! You do this by simply selecting a new SCSI ID starting with 1:x instead of 0:x. Existing VMDK’s should be on SCSI Controller 0. Your RDM’s should be on SCSI controller 1. Although my screenshot shows 0:3 this should read 1:3.
You can now save your VM configuration. Your VM will now access the RAW SATA drives and be able to use things like SMART to monitor its health.
See below; I am adding my three 1TB drive to FreeNAS to create a new ZFS pool.
Stay tuned for an upcoming blog post on FreeNAS and NexentaStor which may or may not put you off ZFS [in a VM] altogether!
- Oct 25, 10
This statement is becoming almost cliche in the blogging world, but this post is more for my own benefit to help me remember some of the work I’ve been doing on MSA lately:
The MSA 2000 series has a pretty dire web interface – when you compare it with other SAN’s – and to make matters work, it mixes and swaps terminology with HP’s own EVA’s (a vdisk on an MSA refers to a RAID array made up of physical disks. A vdisk on an EVA is a virtual RAID array which sits on top of the underlying RAID 0 structure). The web interface is challenging enough as it is when it works, but recently the web page of the management interface has been timing out.
Thankfully SSH access still works and responds well. I’ve been using…
restart mc a
…to restart the management interface which helped things for a short while but ultimately I found myself falling back to SSH more and more.
The following procedure follows the steps I took to rename and prevent a volume to a host.
To give a host a friendly name (from its WWN):
set host-wwn-name host <wwn> <friendlyHostName>
To rename a volume:
set volume <volName> name <new_volName>
To view what hosts a volume is currently presented to and what volumes map to what hosts:
show host-maps [<friendlyHostName>] show volume-maps [<volName>]
To actually present a volume to a host:
map volume <volName> [lun <lunID>] host <friendlyHostName> access rw
In theory the LUN ID can be left blank, but in practise I found that it would not map unless I specified an ID. Don’t forget to map both HBA’s for a host.
After presenting the volume to the Windows 2003 host I was dismayed when the Storage MMC could not see the new lun. Hitting F5, of course, was not enough. On the Disk Management section folder in the left hand side of the MMC window, right-click and select Rescan Disks.
I’ll update thist post if I use and feel the need to remember some other command line options for the MSA 2000 but hopefully we’ll be migrating off it to EVA soon!
- Aug 17, 10
Seems the Netbackup reporting script is in pretty high demand at the moment. Obviously lots of you just as frustrated with NBU reporting as I was! I’m really pleased that so many of you are finding it useful – though I am starting to wonder if letting it out in to the open without a license agreement (a free one, don’t worry) is a good idea.
I have placed the scripts in a public DropBox location which is linked to at the bottom of the original Netbackup Report Scripting post. In the mean time, please see the PayPal donation button to the right now. If you’ve used my scripts and find them useful – and certainly if you plan to use them in a business environment – please consider donation as much or as little as you can.
I also have some nice new blog posts ready to upload as soon as I can sort my image hosting out! Those of you struggling with EVA performance monitoring and tuning, stay…err….tuned!
- May 04, 10
I’ve seen Netbackup used in two completely different ways now. In my last environment all tapes rotas, movements and scratch tapes were handled manually. We had a spreadsheet showing tape sets (see below for an example), when these tapes were collected by Iron Mountain, which days of the months we defined as Monthly/Quarterly backups and so on. In this scenario each set of tapes is a number of tapes – on some days you may use less, on some days you may not have enough loaded.
In the new environment we let Netbackup dictate which tapes are in the scratch pool and free for re-use based on the expiration policy of the jobs written to those tapes. In theory this should mean that you only ever use as many tapes as you need for a given backup job. The problem with this set-up is that although it is, in theory, more efficient, you don’t always know which tapes you will need each night and how many you have free.
Each day guys were running NBU command lines to show which tapes were in the scratch pool, but having to manually sift through them to see which tapes were currently sat in drives/libraries and of the ones held off-site, which tapes were of which format (LTO2, 3 or 4).
The following script can be scheduled to email out a list of tapes in the scratch pool, which are in what library and also groups the offsite tapes by format. It was knocked up in a hurry so isn’t as parameterised or dynamic as it should be, but the comments should help you fix it for your needs…
Here’s the script and HTML header & footer…
scratch_list.ps1
$debugpreference = "continue" #This "output" variable runs and stores the output of the NBU command line #Be sure that this .exe is in your path, otherwise change the below to include the full path to the executable. $output = vmquery.exe -b -pn Scratch $htmlfile = ".\output.txt" $today = Get-Date #Arrays to hold and sort info about our tapes. #NOTE: this script is hard-coded to work for LTO2 (HCART2), LTO3 (HCART3) and LTO4 (HCART) tape drives/libraries [array]$aTapes = @() [array]$aHOffsite = @() [array]$aH2Offsite = @() [array]$aH3Offsite = @() #NOTE: this script is hard coded to include the specific names of tape drives attached (via various media servers) to the master server $aRobots = @{"TLD0" = 0; "TLD1" = 0; "TLD2" = 0; "TLD3" = 0; "TLD4" = 0; "TLD5" = 0; "TLD6" = 0; "TLD7" = 0; "TLD8" = 0; "TLD9" = 0; "TLD10" = 0; "TLD11" = 0; "TLD12" = 0; "TLD13" = 0; "TLD14" = 0} $emailBody = Get-Content "header.html" $aOffsite = 0 #Function to send mail (currently configured to send via Exchange) function sendEmail() { $eServer = New-Object system.net.mail.smtpClient("exchange_server01") $From = "netbackup@yourcompany.com" $To = "nbu_reports@yourcompany.com" $todayDate = [string] $today.day + "/" + [string] $today.month + "/" + [string] $today.year $title = "Scratch tape list for $todayDate" $Body = $emailBody $eMsg = New-Object System.Net.Mail.MailMessage($From,$To,$title,$body) $eMsg.IsBodyHTML = $True $eServer.send($eMsg) } #Take output of NBU command line util, and for each line pick out bits of the string #Chuck everything in to a new array called $aTapes foreach ($line in $output){ if (($line.contains("HCART")) -and (-not $line.contains("A00"))) { $tName = ($line.substring(0, 6)).trim() $tType = ($line.substring(8, 6)).trim() $tRobot = $line.substring(23, 5).trim() #this captures a wide area with some white space that needs trimming $tRobot = $tRobot.replace(" ", "") $aTapes += ,($tName, $tRobot, $tType) } } #Go through the array we just created and then put the contents in to seperate arrays #dependant on whether they are LTO, LTO2 or LTO3 tapes... for ($j = 0; $j -lt $aTapes.length; $j++) { $tapeID = $aTapes[$j][0] $robot = $aTapes[$j][1] $type = $aTapes[$j][2] if ($robot.contains("-")) { switch ($type ) { "HCART" {$aHOffsite += ,($tapeID)} "HCART2" {$aH2Offsite += ,($tapeID)} "HCART3" {$aH3Offsite += ,($tapeID)} } } else {#If a library type does not contain HCART, HCART2 or HCART3 we must assume it's offsite (or in a safe)) $temp = "TLD$robot" $aRobots[$temp] = $aRobots[$temp] + 1 } } #Take the contents of $aRobots (offsite tapes) and put it in to a variables ready to be dumped in to our html output foreach ($element in $aRobots) { $element > $htmlfile } #Start pipping everything out to the $emailBody variable which will make the body of our email $emailBody += '<table><tr><td>Tape drive</td><td># scratch tapes</td></tr>' foreach ($line in Get-Content $htmlfile) { if ($line.contains("TLD")) { $emailBody += "<tr>" $robot = $line.substring(0, 5) $count = $line.substring(31, 3) $emailBody += "<td>$robot</td><td>$count</td>" $emailBody += "</tr>" } } $emailBody += '</table><br/><br/>' $emailBody += "<b>Offsite tapes...</b><br/>" $emailBody += "<b>HCART</b><br>" foreach($element in $aHOffsite) { $emailBody += "$element<br/>" } $emailBody += "<br/><b>HCART2</b><br>" foreach($element in $aH2Offsite) { $emailBody += "$element<br/>" } $emailBody += "<br/><b>HCART3</b><br>" foreach($element in $aH3Offsite) { $emailBody += "$element<br/>" } $emailBody += Get-Content "footer.html" sendEmailAs per my other NBU scripts I use HTML files to store the CSS and opening HTML in order to easily format nice looking emails. See below for examples…
header.html
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"> <html> <head> <title>Scratch tape list</title> <style type="text/css"> body { font-family: Calibri, Arial, Helvetica; color: black; font-size: small; background-color: #FFFFFF } p { font-size: small } .big { font-size: small } table { font-size: small; padding: 0px ; border-spacing: 2px ; empty-cells: hide ; border: 1px solid #AAAAAA } td { background-color: #EEEEEE; vertical-align: top } th { background-color: #EEEEEE; vertical-align: top ; text-align: left} </style> </head> <body> <p>*** This email is now automatically generated ***<br></p>footer.html
<br/> <p>Regards,<br/><br/> </p> </body> </html>
Let me know if this helps you out at all or if you can suggest any improvements…
- Mar 12, 10
Some time ago I started using CrashPlan with CrashPlan Central to finally start backing up my family photos and important documents off-site. It’s been working pretty well, despite my frustrations over slow upload speeds. It’s pretty much a fire and forget solution.
Recently I’ve installed FreeNAS on to an old PC to (a) start getting some stability to my storage solution, and (b) play around with ZFS and iSCSI at home. I tried OpenFiler but it pales in comparison to FreeNAS. I’ve gone against my better judgement and I’m using some of the cooler plugins including the Torrent client, Dynamic DNS and I’ve even installed SABNzbd on it. FreeNAS comes with rsync built in and even Unison which is an interesting solution to cross platform backup/synchronisation, but I’ve payed for my CrashPlan Central plan so want to make use of it!Only problem now is backing up my data!
CrashPlan doesn’t support backing up from a network share – either mapped drive or UNC path. There is a work around on the CrashPlan site which works but is a little convoluted so wanted to post a nice, quick version here.
Read on after the break… The problem is simply that CrashPlan runs as a service using the SYSTEM account. To allow us to backup a network share we have a couple of options….
- Change the service to run as a standard user account (e.g. the one you’re logged in to your machine as)
- Map a drive under the SYSTEM account so that it is visible to CrashPlan
I’d recommend the latter but if you just plan on using this on a desktop machine and/or want a quick and dirty solution, here’s how you change the logon account for the service…
- Open up the Services MMC and find the CrashPlan Backup Service. Double-click it to open its properties
- Click the Log On tab and then select the This account radio button.
- Enter the username and password of a local/domain user account with full permissions to the network share you want to back up
- Click OK and then restart the service
- Relaunch the CrashPlan application and you’ll now be able to see drives that you currently have mapped as a backup selection as well as local drives
The preferred method is to map a drive as the SYSTEM account. That in itself is no big deal, the thing is getting the drive mapped as soon as the machine boots and doing it each and every time your machine restarts. To do this we simply make use of Vista/7′s Task Scheduler.
The batch script, which we’ll save as c:\mount.bat simply uses a NET USE command. The one drawback to this method is that the username and password need to be saved in the script. You must use a username which has permissions to the network share. This need not be a local username, in the case of FreeNAS, if you have it joined to a domain as I do.
Here’s the code:
net use M: \\freenas\media /USER:dave@mydomain.lan myP4ssw0rd
Once you have that saved open up the Task Scheduler…
Create a new task (I keep my tasks in a seperate folder to the default Windows ones simply for easier administration). Give your ask a name, for example Mount drive for CrashPlan. Select Run whether user is logged on or not, tick the Run with highest privileges box and then click Change User or Group…
Type system and then Check Names (it should capitalise and underline). Click OK.
Back in the Create Task window slect the Triggers tab and click New… Select At log on… from the drop down list and click OK to close the window.
Now select the Actions tab and, again, click New… leave the action as Start a program and then browse to your batch that we created above.
Click OK to close the New Action window and again for the Create Task window.
Job done! You can right click on the task and select Run to try it out.
A much more elegant solution would be to have a native FreeNAS (or just BSD) client for CrashPlan or to make use of rsync/Unison. This will do me for now though.
- Mar 02, 10
We had a few issues recently where SnapManager for Exchange and SnapDrive were failing to communicate properly which resulted in a weeks worth of missed snapshots before anyone realised what was going on.
I wrote this script to report on the status of snapshots. The filer name is passed as a variable and a list of volume names is stored as a variable at the top of the script. The output is nicely formatted email.
- The script uses SSH to connect to the filer (see my SSH guide for Netapp filers)
- For each vol listed in the array, the list of snapshots is returned
- Any line with “__daily” in is processed – all others are ignored (we only care about our nightly verified snapshots which are all named __daily)
- The snapshot name contains the date/time it was taken – this is read and if the date is within 24 hours of now a success is return (since reports are run the day after the verified job). If the date is over 24 hours a failure is returned.
- A nicely formated HTML email is sent out which looks a little something like this…
Please see below for today’s backup report.
VOL Name Status Snapshot name Vol_Exc02_DB Success 0% ( 0%) 0% ( 0%) Mar 02 01:44 exchsnap__exchange02_03-01-2010_23.00.14__daily (busy,backup[0],dump) Vol_Exc02_LOG Success 0% ( 0%) 0% ( 0%) Mar 02 04:38 eloginfo__exchange02_03-01-2010_23.00.14__daily Vol_Exc02_DB1 Success 0% ( 0%) 0% ( 0%) Mar 01 23:00 exchsnap__exchange02_03-01-2010_23.00.14__daily Vol_Exc02_LOG1 Success 1% ( 0%) 0% ( 0%) Mar 02 01:27 eloginfo__exchange02_03-01-2010_23.00.14__daily Vol_Exc03_DB Success 1% ( 1%) 0% ( 0%) Mar 02 00:50 exchsnap__exchange03_03-01-2010_23.00.13__daily Vol_Exc03_LOG Success 0% ( 0%) 0% ( 0%) Mar 02 02:06 eloginfo__exchange03_03-01-2010_23.00.13__daily Vol_Exc03_DB1 FAILURE 1% ( 1%) 0% ( 0%) Feb 26 23:00 exchsnap__exchange03_03-01-2010_23.00.13__daily Vol_Exc03_LOG1 FAILURE 0% ( 0%) 0% ( 0%) Feb 26 00:32 eloginfo__exchange03_03-01-2010_23.00.13__daily See the script and HTML files after the jump…
Don’t forget to change the name of your mail server, your sending and recipient email addresses and, most importantly, the location of plink.exe and your DSA keys used to establish the SSH connection (although you could easily modify this script to use RSH instead).
snapReport.ps1 script
# ============================================================================================== # # NAME: Netapp filer snapshot reports # # AUTHOR: David Warburton # DATE: 12/03/2009 # # PARAMETERS: 1. $filer # Specifies the name of the remote filer to connect to. # example: snapReport.ps1 myfiler01 # # ============================================================================================== $debugPreference = "continue" $errorActionPreference = "stop" $filer = $Args[0] $htmlFile = "D:\Logs\Exchange Snapshot Report\exchange_Snapshot.html" $volumes = ("Vol_Exc02_DB", "Vol_Exc02_LOG", "Vol_Exc02_DB1", "Vol_Exc02_LOG1", "Vol_Exc03_DB", "Vol_Exc03_LOG", "Vol_Exc03_DB1", "Vol_Exc03_LOG1") $header = get-content "header.html" $header > $htmlFile $today = get-date function sendEmail() { $eServer = new-object system.net.mail.smtpClient("mailserver.yourdomain.com") $From = "youraddress@yourcompany.com" $To = "backup_reports@yourcompany.com" if ($checkType -eq "weekly") { $title = "Weekly " } else { $title = "Daily " } $todayDate = [string] $today.day + "/" + [string] $today.month + "/" + [string] $today.year $title += "Exchange snapshot Report for $todayDate" $Body = get-content $htmlFile $eMsg = new-object System.Net.Mail.MailMessage($from,$to,$Title,$body) $eMsg.IsBodyHTML = $True #$eMsg.cc.add($cc) $eServer.send($eMsg) } function checkSnap($volName) { $output = D:\Utils\PuTTY\plink.exe svc-ssh@$filer -i D:\Utils\PuTTY\id_dsa.ppk snap list $volName $validsnap = $False $snapname = "" foreach ($line in $output){ if ($line.contains("__daily")) { $sMonth = $line.substring(62, 2) $sDay = $line.substring(65, 2) $sYear = $line.substring(68, 4) $sDate = [datetime] "$sMonth $sDay $sYear" if (($sDate.date).addDays(1) -eq $today.date) { $validsnap = $True $snapname = $line } } if ($validsnap) { return $snapname } } return $False } foreach ($vol in $volumes) { $line = "<tr><td>$vol</td>" $status = checkSnap $vol if ($status -eq $false) { $line += "<td><font color=red>FAILED</font></td>" } else { $line += "<td><font color=blue>Success</font></td>" } $line += "<td>$status</td></tr>" $line >> $htmlFile } $footer = Get-Content "footer.html" $footer >> $htmlFile sendEmailHere are the HTML bits for the header and footer of the email…
header.html
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"> <html> <head> <title>Backup Report</title> <style type="text/css"> body { font-family: Calibri, Arial, Helvetica; color: black; font-size: xx-small; background-color: #FFFFFF } p { font-size: x-small } .big { font-size: x-small; font-weight: bold } table { padding: 0px ; border-spacing: 2px ; empty-cells: hide ; border: 1px solid #AAAAAA } td { background-color: #EEEEEE; vertical-align: top } th { background-color: #EEEEEE; vertical-align: top ; text-align: left} </style> </head> <body> <p>*** This email is now automatically generated ***<br><br><b>Please see below for today's backup report.</b></p> <p> <b>NOTE: </b><br/> - Exchange and SQL snapshots are managed by SnapManager and are alerted independantly to this report.<br/> - Not <u>all</u> snapshots are shown in these reports. Hourly snapshots are currently not reported upon. </p> <table> <tr> <th>VOL Name</th><th>Status</th><th>Number of Snapshots</th><th>Last snapshot taken</th> </tr>footer.html
</table> <br/> <p>Regards.</p> </body> </html>
- Mar 01, 10
Something I just remembered when performing a restore for a user.
When copying files from a CIFS snapshot using the hidden .snapshot directory NTFS permissions are lost. Instead they are inherited from the folder you are copying to.
To maintain permissions, use NDMPCOPY specifying the snapshot name in the source (first) parameter…
ndmpcopy "filer1:/vol/vol_A/qtree_A/.snapshot/nightly.1/MyDocs" "filer1:/vol/vol_A/qtree_A/MyDocs"
You can enclose the path in quotes if the path has spaces in.
- Feb 26, 10
Netapp have released their new version of their Rapid Cloning Utility – a vCenter plugin which allows you to provision and new datastores and clone hosts (including VMware View 4 VDI’s) with ease right inside of the Virtual Infrastructure Client. vCenter 4 is needed but it is compatible with ESX 3.5 and 4.
The great thing is that all the storage processing is offloaded from vCenter and is performed entirely on the filers. I’ve not had a chance to play with the RCU yet but this just looks utterly awesome! Check out this preview blog post and video from Netapp:
If you don’t have time to sit and read, then at least check out the video. I dare you not to be impressed!
Source: http://www.ntapgeek.com/2010/02/netapp-updates-rcu-for-vmware-vsphere-4.html
Do any other storage vendors have similar tools?
-
Recent Tracks
- Blakefield Gold by Sweet Billy Pilgrim
- Archaeology by Sweet Billy Pilgrim
- Joyful Reunion (Album Version) by Sweet Billy Pilgrim
- Kill Me - Radio Edit by Make the Girl Dance
- The World Is Ours by CatCall
- Tchiki Tchiki Tchiki by Make the Girl Dance
- Swimming Pool (Gloves Original Mix) by CatCall
- Swimming Pool (Julian Mendelsohn Edit) by CatCall
- Satellites by CatCall
- The World Is Ours by CatCall
Snow PatrolColdplayElbowNoah and the WhaleMumford & SonsThe All-American Rejects30 Seconds to MarsLinkin ParkAthleteMuseTurin BrakesStereophonicsAlanis MorissetteThe StrokesThe ScriptNewton FaulknerEllie GouldingThe CharlatansTunngShe & HimStarsailorNoel Gallagher's High Flying BirdsJack JohnsonThe WhoPendulumDubba JonnyThe WombatsEmmy The GreatThe xxKings of LeonSpiroDaft PunkTravisRage Against the MachineGorillazEmbraceKeaneThe Temper TrapThe HoosiersSkrillexLawrenceCastBon IverOasisThe King BluesSigur RósThe Vervedeadmau5EfterklangKasabian- 42 plays
- 29 plays
- 24 plays
- 24 plays
- 23 plays
- 20 plays
- 16 plays
- 15 plays
- 15 plays
- 15 plays
- 15 plays
- 14 plays
- 13 plays
- 12 plays
- 12 plays
- 12 plays
- 12 plays
- 12 plays
- 12 plays
- 12 plays
- 11 plays
- 11 plays
- 10 plays
- 10 plays
- 10 plays
- 10 plays
- 10 plays
- 10 plays
- 10 plays
- 10 plays
- 9 plays
- 9 plays
- 9 plays
- 9 plays
- 8 plays
- 7 plays
- 7 plays
- Mirrorball by Elbow7 plays
- 7 plays
- 7 plays
- 7 plays
- 7 plays
-
Reading
-
Mixes
- 2949 plays / Created May 20, 1240 + tracks for the country lovers! Fun diverse mix of the best new and old summer country :) Featuring Artists include Thompson Square, Miranda Lambert and Conway Twittie.
- 10181 plays / Created May 19, 12Must be played LOUD. Some old flair tracks with some new to pump up your summer :) Featuring Artists include The Hollies, The Asteroids Galaxy Tour, and Hall & Oates.
- 22467 plays / Created May 14, 12Fun covers. You might not even recognize the song!
- 32495 plays / Created May 12, 1240 + tracks. Give it a good listen and don't be fooled by the first 20 seconds... I dare ya. It will be worth it! Whether you know these artists or not summer 2012 will be one to remember!
- 63208 plays / Created May 06, 1230 + tracks filled with whole fun lightheartedness to smile and jump around to :)
- 14354 plays / Created May 05, 12Awesome covers by awesome artists. Artists include Ed Sheeran, The Kooks and Regina Spektor.
- 743 plays / Created May 04, 1220+ tracks. Break free and just let go. Featuring Artists include King Charles, Greg Laswell and Iron & Wine.
- 1407 plays / Created Apr 28, 12Just havin a good time :) Primarily country female artists including The JaneDear Girls, Bomshel and Sugarland.
- 9118 plays / Created Apr 26, 1230 + tracks. Designed to free your mind. Featuring Artists include Feist, Bon Iver and Ben Howard.
- 6755 plays / Created Apr 25, 12For those summer nights walking on the beach or those days when you just want to sit back and chill while listening to some reggae. Featuring artists include Rebelution, SOJA and 311.
- 10444 plays / Created Apr 24, 12Full of guilty pleasure tracks for the closet pop lovers. You know you like them :)
- 9976 plays / Created Apr 23, 12So much love for the first one I had to make another one! 20 + tracks. Diverse mix. Featuring artists include Adele, MGMT and The Beatles. Here's the link to the first mix- http://8tracks.com/jbear_4/cello-covers
- 8926 plays / Created Apr 21, 12Sometimes you just need some new country when you're tired of hearing the same songs played on the radio! Featuring Artists include Lee Brice, Kip Moore and Keith Urban.
- 1121 plays / Created Apr 21, 12Tracks to help remind you of why you should enjoy life :) Featuring artists Include The Civil Wars, Ben Howard and Ed Sheeran.
- 17892 plays / Created Apr 15, 1250+ songs SUMMER 2012 BRING IT! Best new and old country. Featuring artists include Luke Bryan, Blake Shelton and Shania Twain.
- 2389 plays / Created Apr 12, 12Someone you can act your complete awkward self with :) 20 tracks. Featuring artists include Nevershoutnever, The Shins and Jason Mraz.
- 14059 plays / Created Apr 11, 12The playlist is from the perspective of a cello. Artists covered include Coldplay, Glen Hansard and Eminem. Here's the link to the second playlist- http://8tracks.com/jbear_4/cello-covers-2-0
- 3319 plays / Created Apr 09, 12Summer guilty pleasure pop tracks :) Featuring Artists include Natasha Bedingfield, Mat Kearney and One Direction.
- 2246 plays / Created Apr 09, 1230 tracks to help you fall asleep, read, or simply have in the background. The choice is yours. Featuring Artists include Sia, The Middle East and Active Child.
- 362 plays / Created Apr 04, 12"We are all born for love. It is the principle of existence, and its only end." Benjamin Disraeli To hold on or not? New music from featuring artists include The Lumineers, Lovedrug and Edward Sharpe & The Magnetic Zeroes.
- 1193 plays / Created Mar 29, 12Possibly better than the originals. Artists covered Include The White Stripes, Smashing Pumpkins and Gotye.
- 18639 plays / Created Mar 27, 1230+ of the best country summer songs! Perfect for a tailgate, campfire or workout. Featuring artists include Luke Bryan, Sugarland and Tim McGraw.
- 1439 plays / Created Mar 25, 1260+ tracks. Summer happy vibe. Perfect for a road trip! Featuring artists include Girls, The Shins and The Naked and Famous.
- 1031 plays / Created Mar 24, 12Second playlist for 2012 country. It won't disappoint :) 16 tracks. Featuring Artists Include Chase Rice, Dierks Bentley and Luke Bryan.
- 611 plays / Created Mar 24, 12Foot stomping hand clapping songs designed to put a smile on your face :) Featuring artists inlude Jack White, The Paper Kites and The Head And The Heart.
- 4296 plays / Created Mar 22, 1230 soft acoustics tracks. Ideal for study seshes, rainy days and a sweet nap. Featuring artists include James Vincent McMorrow, Joshua Radin and Trevor Hall.
- 866 plays / Created Mar 19, 12A ruffled mind makes a restless pillow. ~Charlotte Brontë A relaxing mix of classical piano music to free your mind :)
- 383 plays / Created Mar 15, 12The best of Kid Cudi. Including REMIXES.
- 3012 plays / Created Mar 14, 12ALL of the songs on The Hunger Games Soundtrack! "May the odds be ever in your favor."
- 572 plays / Created Mar 11, 12Made to put a smile on your face :) A few of my favorite new indie tracks for summer. Featuring artists include Imagine Dragons, Walk the Moon and Lovedrug.
- 12949 plays / Created Mar 09, 12New 2012 country mix for the warm weather :) Featuring artists inlude Dierks Bentley, Luke Bryan and Adam Gregory.
- 237 plays / Created Mar 05, 12Some sweet folk male vocals to add to your day! Featuring artists include Marcus Mumford, Ben Howard and Jacko Hooper.
- 496 plays / Created Mar 01, 12When you just want summer to come now! Enjoy! Featuring Artists Keith Urban, Carrie Underwood and Dierks Bentley.
- 576 plays / Created Feb 23, 1214 tracks. Covering Drake, Foster the People and Bon Iver. Enjoy!
- 436 plays / Created Feb 20, 12Nothing like country music to make you smile. Featuring artists: Luke Bryan, Jake Ousley and Scotty Mccreery.
- 325 plays / Created Feb 17, 12Songs that make you smile and want to just dance! Featuring Artists Walk the Moon, Young the Giant and The Black Keys.
- 679 plays / Created Feb 12, 12better than the originals. Covers from Lana Del Rey, Adele, to Rihanna and Nicki Minaj. Just listen!
- 834 plays / Created Feb 11, 12Somethin' bout Chris Young, elenowen and keith urban make you love country music.
- 18576 plays / Created May 17, 1220 songs I love to start my day off to and help me wake up in a good mood..
- 21885 plays / Created Apr 28, 12Over 3 Hours of ambient, instrumental, post-rock that for me, is perfect for studying when the stars come out & I need to stay awake until the sun rises.. I hope it does the same for you.. [Headphones recommended!] Part 2 up now! - http://8tracks.com/cosmicapathy/it-s-late-focus-pt2 (another 3 hours+)
- 15692 plays / Created Mar 22, 12“You know, I dream sometimes about flying. It starts out like I’m running really, really fast and I’m like superhuman and the terrain starts to get really rocky and steep and then I’m running so fast that my feet aren’t even touching the ground and I’m floating and it’s like this amazing, amazing feeling… I’m free, I’m safe. Then I realize I’m completely alone. And then I wake up.” if you like this mix, follow me! or atleast check out some of my other mixes, i'm always creating new ones. :)
- 104106 plays / Created Mar 23, 12Fact 1: Without decent music there's no way you'd get through your homework, essays and revision. Fact 2: Songs with lyrics distract the part of your brain which you use to repeat things to yourself when you're learning. The Solution: here's 45 fantastic lyricless compositions, film soundtracks and piano covers to get you through it. Good luck! Full track list and free download available at http://sarahtheotaku.wordpress.com/2012/04/16/the-science-of-studying-track-list-and-download-link/
