Back To Basics: Creating Virtual Machines

15 Apr

It might seem odd at this stage to have section introducing virtual machines. Ideally, this something you already know something about – or else why would you be here? But its perhaps worthwhile taking a little history lesson about the development of the virtual machine since VMware became to gain mass recognition in the 2003/4 period. Back then the hardware capabilities of the VM were more modest. The maximum number of vCPUs was just two, and the maximum amount of memory was just 3GB. On the virtual disk side of the house, the VMDK maximum size was a merely 2TB. Fast forward to the current period the scale of the VM has increased massively – to the degree that the only barrier to virtualization is whether it is economic. Nowadays, a VM can have up to 64 vCPUs, 1TB of RAM, and 62TB virtual disks. Advancements have also introduced the ability to give the VM direct access to hardware, where IO performance demands require it.

Upload Operating System ISOs to a Datastore (Web Client)

Perhaps the most common way of installing a Guest Operating System to a VM is using the original CD-ROM/DVD .iso that is used to distribute it. Being able to download .ISO images of Operating Systems has become the norm in the period of high bandwidth internet, and certainly beats having to order physical media deliveries. A VM can optionally boot to PXE, and therefore other OS deployment tools are available such as the Ultimate Deployment Appliance (UDA), Windows Deployment Services and third-party tools. However, it remains the case that most virtualization admins use a DVD .iso

Read the rest of this entry »


Travails in Hyper-V R2eality: Tales of the Red Mist: A Messcellany

10 Apr

fistmonitorDo you geddit? Miscellany and Messcellany. It’s wots called a pun my friend.

In my travails in Hyper-V R2eality, I’ve come across of number of facepalm moments – and also a number “I want to drive my fist through the screen” situations. Sadly, none of these individual add up to much on their own. Collectively they are more than the sum of their parts. I wanted to share those experiences with you – and I thought a “messcellany” might help facilitate that.

The kind of thing I’m getting at is the daily grind of not-workingness that you have to live with. Do you ever have days, weeks or months (years!) where everything seems broken, and you feel your will to live is slowly being sapped by software. That’s the kind of thing I’m thinking about. At all times you try to remain calm. Deploying all those strategies your therapist taught you in “Anger Management” classes. Perhaps I’m getting old and jaded, but my personal well of patience for things that don’t work is one that appears to get shallower and shallower as the years roll by. You would think after many years of working in IT, my levels of hope would have levelled off to such a low-level of expectation – that I would no longer experience the feeling of disappointment. Sadly, the battle scars of working in IT haven’t eviscerated all semblance of optimism. And I still have this crazy notion that stuff should work – all the time, and never break. I know I need my head examining…

Anyway, for the sake of my own sanity, I’ve tried to approach these with a big fat dose of humour – and some videos to amuse along the way… mainly on the theme of Monty Python sketches…

  • To P2V or Not P2V
  • When is an upgrade not an upgrade?
  • Network Virtualization “Gateway” –  Clusters in clusters…
  • Remember NOT to store stuff in the C: drive
  • Adding Windows Hyper-V Host to SCVMM
  • A State of Job #Failure
  • Prerequisites: No one expects the Spanish Inquisition
  • It’s not Hypervisor, its naughty, naughty boy
  • The RunAS Olympics…
  • I tell you what’s wrong with it. It’s dead…

Read the rest of this entry »


Performance Stress Tools

04 Apr

Screen Shot 2014-03-28 at 17.13.26One of my new projects is to get my head round VMware Operations Manager – or VCOPS as its more commonly known. I’d freely admit that performance is one of my weak areas. I’m pretty good at troubleshooting and resolving any number of configuration problems, but resolving performance problems isn’t one of my strengths. Why?

Well, I’ve spent a great many years living in the abstract world of the lab whether that be as trainer, author or now at VMware. In that time I didn’t a get a whole lot of exposure to genuine performance issues – after all a lab environment never experiences the same non-linear performance issues that you see in the real world. What did happen a lot of the times was that students would bring me their performance problems. And I would from principles try to diagnose them. By first principles – I mean things such as:

  • Over-use of SMP vCPU
  • Disk intensive VMs placed on the same LUN/Volume/Spindles
  • The wrong RAID level used
  • Insufficient RAM allocated
  • Miss-use of various features in vSphere such as poor resource pool design, inappropriate shares and so on

Despite my lack of exposure, one of my favourites parts of the Install & Configure/Fast-Track course were things like Limits/Reservations/Resource Pools/DRS/ESXTOP and so on. Mainly, because these are really juicy topics that can be tricky to explain. I enjoyed the challenge.

So now I’m looking at VCOPS I’m looking at this subject all over again, whilst bearing mind that vCOPS isn’t merely or just a performance monitoring solution. It actively, or rather pro-actively goes looking for “health problems”. There’s two analogies for this. See yourself as Dr VI Admin MD at vSphere Hospital for the Virtual Machine if you like. What VCOPS is giving the pre-emptive, pro-active diagnostic tools to analyse your patients (the VMs), and deal with their minor symptoms before they are really unwell. Another analogy I like is the “dashboard” in your car. Not only does tell you speeds and feeds – there’s also a little red light that gets illuminated when your running low on oil. It’s better to receive pro-active, pre-emptive alerts then be at the side of the road with seized engine.

So anyone thing I’ve been looking at is different tools for generating a fake workload inside a VM – for the four core resources of CPU, Memory, Disk and Network. I thought if I put together a compendium of tools in a single web-page, it might help someone looking to do the same thing.

A couple of really strong observations have come out from my early use of VCOPS. Firstly, vCenter and other 3rd party performance monitoring tools tend to just contain “thresholds” offer a simple “traffic light” view of performance. You know the same tedious green, yellow, red system where alarms are triggered at 75%, 90% and so on. That’s all well and good. The trouble is these 3rd party tools in the main aren’t really showing deviations. So if an application grows in resource usage (say CPU) over a 6hr period from 10%, 20%, 30%, 40%, 50% and 70% – most of them won’t tell you anything. Until the VM has smashed through one of their pre-configured “thresholds” such 75%. By then I think you could argue that problem has got out of hand. Wouldn’t it be far better to alert the administrator to underlying, under-the-surface, iceberg of a problem – rather than waiting for the tip of the iceberg to appear just above the water? I see this very much like a Doctor looking for diagnostic information about the health of patient. When the Doctor monitors the patient they look for tools that can show that a change is taking place, something other than normal. The other thing I’ve liked about VCOPS is how its identified a number of problems in the build of my vSphere homelab. Sometimes those problem have been my own making, other times VCOPS has identified problems in vSphere that has lead to know some known issue in a KB. That for me shows two benefits. VCOPS isn’t just about performance, its about configuration – or should I say “Operations” (the clue is in the name of the product after all!), secondly I can absolutely trust what it telling me – it doesn’t try to pretend that everything is right in the world when it isn’t.

Anyway – less of my ramblings – to the tools overview…. which I’ve catagorized as CPU/Memory, Disk/Network IO, and Application Tools.

Read the rest of this entry »

No Comments

Posted in vCOPS


Thank you!

02 Apr


Just a quick little post to say thank you to all the people who voted for me into the Top10 Bloggers poll, and to also announce that for another consecutive year I’ve been granted the vExpert Award for services to the community. It’s been an interesting year. Having sold my old site RTFM Education some time ago, and last year being the first real year of I’ve had to work incredibly hard to maintain and (yes) regain readership. All in climate very different from when I started writing about VMware in 2004…

I think these polls and awards are great way for the community to show their appreciation for the largely free work that VMware’s Unpaid Army of Evangelists engage in (admittedly, I am on the payroll!). Despite my recognition I often wonder if the stuff I’m pumping out is of interest – and hit counts and such like – don’t really reflect how useful people find the content. I’m sure plenty of other bloggers wonder if what they doing is great, but only they appreciate it. I’ve always found it inspiring when folks come up to me at VMUGs or VMworld and thank me for my work – it gives me the inspiration to continue.


Back to Basics – vCenter Server Appliance

18 Mar

This post is a bit of funny one – as it really isn’t in series with the rest of my “Back to Basics” posts. It’s more “Let jump in the TARDIS, and go back to beginning”, shades of gravitational waves of the Big Bang. I decided to opt for the installable version of vCenter in previous posts because that’s what a lot of folks still use. But for my nested ESX environment I thought it would make things more “complete” if I deployed the vSphere5.5 edition of the vCenter Server Appliance (VCSA). The process is relatively straightforward. There are one or two anomalies. Firstly, if you want to assign a static FQDN/IP address you have to cancel the opening wizard after accepting the EULA. It’s possible to crank up the wizard again once that change has been made, but in the end I thought the manual steps were just as quick and easy. My second anomaly is how well the VCSA responds to FQDN/IP changes after it has been setup. I’m afraid to say I personally struggled with that – so I think if I was speaking to students like I used to, I’d recommend setting an FQDN/IP that probably never change that way you don’t have issues. One thing I found if you do go around renaming and re-ip-ing the VCSA is it has a knock on effect on the “Lookup Service”. I’ve seen that before a couple of times – when the postgres DB filled the VCSA partition in the 5.1 release.

Anyway, for sake of completeness here’s my back to basic post on the VCSA. Enjoy!


This blogpost has recently updated with a “Show Me How” video, which demo’s the import and post-configuration of the VCSA…

Native Quality


VMware vCenter is the companies core management platform, and its common for other technologies both from VMware and third-parties to use it as the central point for accessing ESX hosts and clusters, as well accessing VMs and other components upon which they can add value or further orchestration. At the time of writing VMware support two version of vCenter – an installable version which runs on the Microsoft Windows platform and virtual appliance edition which runs on the SUSE Linux platform. In terms of their core APIs, the two editions are functionally the same. However, there are some features that are for the moment only available on the Windows editions. Historically, third-party add-ons have run as Windows plug-ins. Finally, whilst the scale of vCenter Server Appliance (VCSA) has increased in recent releases, it has lower configurable maximums. In production environments it is the Windows edition that predominates – and this is combination of history (the first version of vCenter was Windows only) and compatibility. With this said, the VCSA is considerably easier to deploy and update/upgrade than the Windows edition – and for this reason it might be more suitable for a small-medium sized deployment where “core” virtualization is all that is required.

Read the rest of this entry »

Comments Off

Posted in BackToBasics


Back to Basics – Creating and Managing a Distributed Switch with PowerCLI (9 of 9)

17 Mar

I think this is my post about Distributed Switch in my “Back to Basics” series. At the back of mind I know I’ve yet to cover “Port Mirroring” as feature, and I will someday double back to that. Right now, I think I have an issue with my physical switch which is preventing me from doing the work on that properly. Usually, at the end of series of post I wrap up with an overview on how carry out similar admin tasks from PowerCLI – and that’s what this post is all about.

Screen Shot 2013-10-21 at 14.01.16.png

Creating an Distributed Switch

The PowerCLI cmdlet New-VDSwitch can create a Distributed Switch in this case the command creates the switch but also enables LLDP in both directions, together with an MTU of 9000 with support for two Distributed Uplinks. Distributed Switches created without the -version parameter will automatically use the most recent iteration of the Distributed Switch.

New-VDSwitch -Name “DSwitch0-GoldCluster01″ -Location “New York” -LinkDiscoveryProtocol “LLDP” -LinkDiscoveryProtocolOperation “Both” -MTU 9000 -NumUplinkPorts 2

Read the rest of this entry »

Comments Off

Posted in BackToBasics


Back To Basics: Migrating from Standard Switches to Distributed Switches (8 of 9)

13 Mar

Note: Currently the desktop vSphere Client supports a drag-and-drop move of multiple VMs from one portgroup to another – however, the Web Client does not. Personally, I think its easier and faster – however, there’s no validation about what the consequences are for those drag & drops…

Generally, it is recommend to starting using Distributed Switch from day one for virtual machines – so as avoid a migration process altogether. However, this may not possible due to licensing restrictions or the fact that using Distributed Switches was not included in the original design at deployment phase. Moving VMs from a Standard Switch portgroup to Distribute Portgroup can be achieved in a bulk way using the Web Client. However, its perhaps wise to test the configuration before embarking on a migration, and to claim a maintenance period for the effected VMs. Therefore should an outage occur then no end-users or customers would be affected. It is also possible to migrate the VMkernel networking away from a Standard Switch if necessary.

Migrating VMs from Standard Switch to a Distributed Switch


This blogpost was recently updated with a “Show Me How” video which demonstrates the process of migrating from a Standard Switch to the Distributed Switch….

Native Quality

Note: This process can also be used to return VMs back to the Standard Switch portgroup if necessary.

Migrating VMs from a Standard Switch portgroup to a Distributed Switch Portgroup can be achieved using a specific option within the “Add and Manage Hosts” wizards.

Screen Shot 2014-02-14 at 14.11.24.png

However, there is a dedicated wizard designed to “Migrate VM(s) to another network” which may be easier for those wishing to move large numbers of VMs from one portgroup to another.

Read the rest of this entry »

Comments Off

Posted in BackToBasics


Back to Basics: NIOC (Part 7 of 9)

12 Mar

Network I/O controls together with Network Resource Pools allow for a sophisticated management of all the traffic types available on a Distributed Switch. There are number of built-in Network Resource Pools, as well the capacity to create user-defined pools. There built-in pools support the following traffic types:

  • Fault Tolerance
  • iSCSI
  • NFS
  • vMotion
  • Management
  • vSphere Replication (VR)
  • Virtual Machine

The settings on these “System” Network Resource Pools can be modified – and no work has been done to assign them to a portgroup. VMware ESXi automatically recognises the traffic type leaving the host, and assigns the settings. By default all traffic is treated equally, except for Virtual Machine traffic which is given a share value of “High”.

Screen Shot 2014-02-20 at 14.48.44.png

These default settings can be adjusted to affect the traffic.

Optionally, user-defined Network Resource pools allow the administrator to define their own pools – and configure the custom settings. This consists of a proportional “share” value. This is a settings is usually configured with parameters of Low, Normal, Medium and High. A custom option allows allocation of a number such as 100. The label or numerical allow to set a high allocation of resources dependent on contention. Under normal operations when network resources are not scarce, the traffic types are able to use as much of the bandwidth available. However, when contention kicks in because of a lack of bandwidth the share system guarantees a predictable outcome. Typically, administrators allocate a higher share value to mission critical traffic such as iSCSI, NFS, Management and Virtual Machines – whereas ancillary traffic which is more expendable and used in background process such as vMotion or vSphere Replication. In addition to the share allocation, these built-in network resource pools an allocation of bandwidth can be granted per traffic type.

Read the rest of this entry »

Comments Off

Posted in BackToBasics


Back to Basics: Configuring NetFlow (Part 6 of 9)

11 Mar

From monitoring it is possible to enable the NetFlow protocol on the portgroup. Once enabled, you can configure the NetFlow collection IP address and TCP Port number on the Netflow Settings of the Distributed Switch. By default NetFlow is disabled on the Distributed Portgroup.

To use NetFlow you will need a collector that gathers the statistics generated by the NetFlow protocol. There are number of free open-source Netflow Collectors as well as commercially available ones as well. This section uses a 30-day evaluation of Scrutinizer NetFlow & sFlow Analyzer from Plixer. They also have a free edition with some feature limitations. It Supports unlimited interfaces on up to 5 routers and stores data for just 24 hours. They also have a Virtual Appliance Edition for which you can apply for a license key.

1. Start by modifying the NetFlow Settings on Distributed Switch underneath the Manage tab, and Edit settings

Screen Shot 2014-02-14 at 11.02.28.png

2. In this configuration box you will need to set the IP address of the collector, together with the TCP port number the NetFlow Connector listens on for NetFlow traffic - typically Scrutinizer listens for devices on ports 2055, 2056, 4432, 4739, 9995, 9996, 6343 by default. The second IP address is a management address assigned to the Distributed Switch (this is not the IP address of the physical switches to which the VMware ESXi hosts are connected).

3. Next configure your Advanced Settings which control the rate of collection.

Screen Shot 2014-03-10 at 13.42.57.png

4. Finally, enabled the NetFlow Setting on the properties of a Distributed Portgroup

Screen Shot 2014-02-14 at 11.03.02.png

5. After a short while the NetFlow Collector should start receiving information from the distributed in the case of Scrutinizer its possible to create groupings – to gather your various Distributed Switch into single view separating from other devices reporting NetFlow information.

Screen Shot 2014-03-10 at 13.50.40.png

From the Reports within Scrutinizer node under the registered Distributed Switch its possible to run reports that cover different time periods and traffic types. For instance this 24-hour report shows traffic between various VMs configured for the Distributed Switch.

Screen Shot 2014-03-10 at 13.53.32.png


Back to Basics: Health Check (5 of 9)

10 Mar

Health Check allows vSphere to analyse the attributes of the Distributed Switch, and compare them to the attributes of the physical switch. It can be use to highlight misconfiguration such as invalid MTU sizes, VLANs, NIC Teaming and failover. For instance an alarm can be raised when the Distribute Switch refers to a VLAN on a portgroup, which isn’t available at the physical layer. By default Health Check is not enabled.

1. Select the Distributed Switch, and click the Manage Tab

2. In the Settings column, select Health check

3. The Edit button to enable the feature.

Screen Shot 2014-02-13 at 13.42.16.png

Viewing Health Check Status

1. Select the Distributed Switch, and click the Monitor tab

2. Select the Health Column

3. Problems will be flagged with a yellow exclamation mark on each affect host.

Screen Shot 2014-02-13 at 13.49.12.png

In this case flagging up that although VLAN101 is available, VLAN102 is not accessible to either vmnic2 or vmnic3 on the host. With further investigation it became clear that whilst VLAN101/103/104 had the vmnic2/3 of each host enabled for tagging – VLAN102 had not been enabled for VLAN Tagging. In error the FT Portgroup was not enabled for VLAN was attemtping to use VLAN0 which did not exist.

These issues were resolved to and the health check refreshed like so – with VLAN103 being used for the FT communication.

Screen Shot 2014-02-13 at 13.58.01.png

Comments Off

Posted in BackToBasics