Glo Networks Technical Blog (Glo Blog)

For those of us Exchange admins who aren’t fully up to date with Exchange 2010 service packs (for whatever reason – don’t berate us) and have slowly been getting annoyed with an ever increasing amount of mailboxes that get automapped in Outlook 2007 and 2010 can rejoice. One of our staff spent 10 minutes reearching, understanding and writing a bit of Powershell to resolve the issue en-bulk.

Just run the following under a Powershell instance with the Exchange modules loaded (i.e. the Exchange Management Shell).

foreach ($u in $(Get-User)) { Write-Host "Clearing the msExchDelegateListLink for $($u.distinguishedname)"; $ad = [adsi]"LDAP://$($u.originatingserver)/$($u.distinguishedname)"; $ad.msExchDelegateListLink.Clear(); $ad.SetInfo(); }

Be aware that if you do this, all auto mappings for all users will get removed. This may not have the desired behaviour as some users may be relying on an automapped account. It should be feasible to alter this as required, as it’s pretty damn simple!

As with all things, do this at your own risk, we accept no liability, yadda yadda yadda.

If you’re curious (and lets face it, if you’re modifying the Active Directory en bulk, you should be!), basically what this does is loop over all users, and clear all values against their Active Directory account msExchDelegateListLink attribute. This attribute is where the automapping gets written to.

As an example imagine 2 users, Bob and Alice. Bob is an Exchange admin and at some point was granted full access (with permission, naturally) to diagnose a problem with Alice’s mailbox without having to disturb her too much directly. Bob will now be added to Alice’s msExchDelegateListLink attribute. It’s this entry that needs to be removed.

Apparently in SP2 there is the facility to disable automapping, at the time of adding the relevant permission; the shiny new “-AutoMapping” argument to Add-MailboxPermission. As with many things Exchange 2007 and newer, it’s just a facility available at the Powershell console, for now. And you know what? We’re ok with that.

Posted by Karl in Applications, Cool Stuff, Scripting, Servers | No Comments »

In the virtualisation game VMWare are big competitors.  They’ve been selling virtual machine software since 1999 and their products are the ‘go to’ virtualisation tools for many businesses (not us at Glo). But since announcing their most recent price structure changes VMWare have experienced a huge amount of criticism from their customer base.

And we can understand why. The basic gist of the change is a cap on the RAM you can apply to your virtual machines per license. Formally licenses were required on a per processor basis alone, now, if you reach the virtual RAM cap for the number of processor’s you have licensed, you will require extra licenses to cover any additional RAM. This increase caused the change to be dubbed the ‘Memory vTax’.

When VMWare first announced this pricing change the memory allowance per license were rather low, meaning (obviously depending on the configuration of the virtual machines) some VMWare customers were looking at their licensing costs being several times what the old pricing structure would have cost. Reacting to the complaints of their customers VMWare have now raised the cap, which should keep the license costs to a more reasonable level for most customers.

Here at GloNetworks we’ve always tended towards the Microsoft Virtualization software ‘Hyper-V’ over the VMWare options, and right now we’re more confident than ever in our choice. It could be argued that WMWares virtualisation software is more ‘feature-full’ however we feel that Hyper-V’s pricing has always been more appropriate for us and our customers’ requirements. And since Microsoft have appeared to confirm they have no plans to use a similar ‘Memory Tax’ in its next Hyper-V product (Windows Server ‘8’ Hyper-V) we’re sure this will continue to be the case.

Posted by James in Applications, General, Operating Systems, Servers | 1 Comment »

As promised in “Our New Private Cloud Platform“, I’m about to divulge all our secrets. Or at least some of them. In vague deal.

I should warn you that this is a blog post aimed at technical people, who have some knowledge of Hyper-V clusters already, so if you’re looking at this from a “users” point of view you may get very lost, very fast. I’m not going to explain every little detail, because quite frankly we’d be here all day.

I feel I should start off by defining our “Private Cloud”. Cloud is term that thrown about a lot recently by marketing staff, and for that reason technical staff need to use it in front of boards and in front of the decision makers. To us techies it may be frustrating, however it’s the world we live in. If you’re uninitiated it’s a very broad term that covers:

• Infrastructure as a Service (IaaS) – Server and networking hardware, possibly server OS, such as Amazon’s AWS,
• Software as a Service (SaaS) – Software provided by a remote system, such as Google Apps (Mail, Docs, etc.), or Dropbox,
• Platform as a Service (PaaS) – Normally software infrastructure for developers to rapidly build software, such as force.com, Google’s AppEngine, or parts of Azure.

Whilst we also provide SaaS products, in this instance our small “Private Cloud” we see as an IaaS offering. It’s a small, 2 node, Hyper-V cluster that we use to run our own and customer’s systems.

As a rough outline our cluster consists of:

• 1x HP Procurve 2810-24G as our main switch,
• 1x Juniper SRX210 acts as our firewall and gateway device for some portions of our network,
• 1x IBM x3250 M3 acts as our physical Active Directory Domain Controller, and also hosts our Data Protection Manager (DPM) 2010 and System Center Virtual Machine Manager (SCVMM) virtual machine under Hyper-V,
• 2x IBM x3550 M3 act as our Hyper-V nodes,
• 1x IBM DS3512 acts as our shared storage,
• 1x QNap TS-459U+ acts as our short term backup storage,
• Several USB hard drives for off-site backup that are routinely swapped.

We’re aware that there are some issues with this design; single switch, single firewall and only 2 Hyper-V nodes. However the importance here is why we chose some of these things and why we don’t care as much right now (this was a significant investment for our small company);

1. Granted all hardware does die. In the event that a switch does we can get one on-site reasonably quickly if we needed to, however we’re yet to have a HP Procurve die on us since we’ve started business,
2. Single firewall is something that we worry about, but we’ve chosen Juniper as they are easily clustered,
3. Provided that you don’t over subscrbe 2 Hyper-V Nodes should be sufficient, however additional nodes can be introduced to the cluster easily in the future.

So whilst we are aware of the problems, I believe that we’ve engineered the system in such a manner that we’re able to introduce new hardware easily, upgrade the existing hardware, and provide some additional redundancy, including multiple switches with multi-chassis LACP links.

We’ve not built this system to compete with Amazon’s amazing AWS, however we have built it with 3 goals in mind:

1. Extensibility,
2. To use as a small reference design,
3. To virtualise our own systems more redundantly. The fact we’re able to host customer’s systems as well is a nice perk.

I won’t take you through the process of setting up your Hyper-V cluster, but I will cover a few bits and bobs that we feel a techy should be aware of before walking into a project like this, but might forget when looking at the big picture.

Clustered Shared Volume, or CSV, is the magic that makes the shared storage work. It’s a clever file system that allows multiple nodes to share the same storage. We’re yet to deploy a CSV using FC so we’re unsure if this is true for FC as well, however in the instance of both DAS and iSCSI what happens is the following;

1. The master node takes control of the storage,
2. All other nodes are notified of this, and effectively redirect all storage requests for the shared storage to the master node, over the network.

It should be clear from this that your choice of network card and switch are very important.

CSVs are not supported by Microsoft for any other use other than Hyper-V clusters. So don’t go getting any ideas.

Jumbo frames on your networking gear is a must. Generally speaking a Jumbo Frame is any ethernet frame that exceeds 1500 bytes, however they’re commonly also used as a naming convention for frames of 9000-9600 bytes (+/- 14 bytes for the header, depending on your switch(es)/NIC configuration language). If you don’t remember how IP and ethernet interact I suggest you go and refresh your memory very quickly. You should recognise the importance of having Jumbo Frames enabled very quickly; it should provide higher performance in situations where large payloads are being transmitted frequently.

At present we’re using Microsoft’s DPM 2010 to backup. The major gotcha that we didn’t see was that DPM 2010 on a Domain Controller is basically a no-no:

For a DPM server that is installed on a domain controller, only protection of data sources local to the DPM server is supported. You cannot install agents on other computers to configure protection.

SCVMM (System Center Virtual Machine Manager) 2008 R2 needs some polish. We’ve had to dive into the database once already. Don’t be afraid of it.

Other than that the project went exceedlingly smoothly. There are a few features that I wish Hyper-V had, in comparison to VMWare and Xen. And I really do wish that there were more, cheaper, graphics cards out there for RemoteFX.

However, that’s just something to plan for as a future project.

Posted by Karl in Applications, General, Hardware, Operating Systems | 2 Comments »

Sometimes SCVMM 2008 [R2] has a problem, and it becomes out of sync with the underlying Hyper-V cluster. Usually a quick refresh of the cluster and you’re stored.

Sadly this is not always the case. In the instance where you see duplicate virtual machines you have a bit of an issue on your hands as you can’t migrate that virtual machine to the Hyper-V cluster node that also has the “missing” version of itself.

If you delve deeper, and check the Failover Cluster Manager, you should see that there is only one copy. With that verified the answer is to break out the SQL to fix SCVMM.

1. Ideally you’ll want to stop the SCVMM service (VMMService)
2. Download and install SQL Studio Manager, or if you’re really hardcore use osql. If you’ve used the default database then you’ll want to connect to COMPUTERNAME\MICROSOFT$VMM$. Otherwise it’ll be where ever it was installed.
3. Now there are “2″ solutions we’ve found for this problem. As always backup your database first.
   • We found by digging into the database that the virtual machines instances are stored in tbl_WLC_VObject and have an ObjectState equal to 220. Deleting these very quickly removes the VM. However it does potentially leave other references behind.
   • The technet way, which we’d recommend, (see http://technet.microsoft.com/en-us/library/ff641854.aspx), which looks at the same table, however also searches all other tables for references.
4. Restart the SCVMM service and you should be laughing all the way to the bank

Just incase Technet ever decides to go away, we’ve put the script from the page referenced above, below.

BEGIN TRANSACTION T1

DECLARE custom_cursor CURSOR FOR
SELECT ObjectId from
dbo.tbl_WLC_VObject WHERE [ObjectState] = 220
DECLARE @ObjectId uniqueidentifier
OPEN custom_cursor
FETCH NEXT FROM custom_cursor INTO @ObjectId
WHILE(@@fetch_status = 0)
BEGIN
DECLARE vdrive_cursor CURSOR FOR
SELECT VDriveId, VHDId, ISOId from
dbo.tbl_WLC_VDrive WHERE ParentId = @ObjectId
DECLARE @VDriveId uniqueidentifier
DECLARE @VHDId uniqueidentifier
DECLARE @ISOId uniqueidentifier

OPEN vdrive_cursor
FETCH NEXT FROM vdrive_cursor INTO @VDriveId, @VHDId, @ISOId
WHILE(@@fetch_status = 0)
BEGIN
DELETE FROM dbo.tbl_WLC_VDrive
WHERE VDriveId = @VDriveId
if(@VHDId is NOT NULL)
BEGIN

DELETE FROM dbo.tbl_WLC_VHD
WHERE VHDId = @VHDId
DELETE FROM dbo.tbl_WLC_PhysicalObject
WHERE PhysicalObjectId = @VHDId
END
if(@ISOId is NOT NULL)
BEGIN

DELETE FROM dbo.tbl_WLC_VCOMPort
WHERE HWProfileId = @ObjectId
DELETE FROM dbo.tbl_WLC_HWProfile
WHERE HWProfileId = @ObjectId
DELETE FROM dbo.tbl_WLC_VMInstance
WHERE VMInstanceId = @ObjectId
DELETE FROM dbo.tbl_WLC_VObject
WHERE ObjectId = @ObjectId
FETCH NEXT FROM custom_cursor INTO @ObjectId
END
CLOSE custom_cursor
DEALLOCATE custom_cursor
COMMIT TRANSACTION T1

Posted by Karl in Applications, Servers | No Comments »

Microsoft have announced they are updating pricing for Hosted BlackBerry Service for Exchange Online. Soon this service will be available free of charge to BPOS subscribers (new and existing).  The Hosted BlackBerry service offers increased functionality compared to the other, already free option: BlackBerry Internet Service.

Previously BPOS users who access their email via the free BlackBerry Internet Service were limited to wireless email access, pushed from the mobile operator. Other information, such as contacts and calendar data had to be updated through a wired connection.

Hosted Blackberry services allows wireless access to e-mail, calendar, tasks and contacts with global address list (GAL) integration, and device management such as device wipe and password reset.

By scrapping the current Hosted BlackBerry services pricing structure and moving towards a free service for BPOS subscribers, Microsoft will be hoping to see businesses that have come to like and possibly rely on their BlackBerry looking on the BPOS (and Office 365 in the future) offerings more favourably.

Posted by James in Applications, General, Small Business Software | 1 Comment »