Virtualizing databases has huge financial and operational benefits – in particular with Oracle, where physically deployed database servers are typically heavily under-utilized which leads to huge over-spending on license cost.
Of course poor efficiency on database servers leads to higher processing requirements, to higher number of CPUs purchased, and in turn to massive additional license and maintenance revenue for the software vendor.
No surprise that software vendors attempt to stop or delay efforts to reduce poor efficiency in any way they can, using all the tricks in the playbook plus a number of dirty tricks that you will never find in books on business ethics.
The latest roadblock Oracle has come up with is what we’ll refer to as the VMotion trap.
Disclaimer: I will not be liable for any false, inaccurate, inappropriate or incomplete information presented in this post. If you want to use the information in this post, verify the legal implications yourself or with advise from an independent, specialized 3rd party.
What does it mean? With the introduction of VMware Vsphere 5.1, virtual machines can be moved online (with VMware VMotion) to other VMware clusters. This is a change from previous versions where VMotion was limited to movement within a VMware cluster. Customers deploying Oracle on VMware usually would license only the physical cluster where they would run Oracle and keep non-Oracle applications on non-licensed servers.
Oracle now claims that all physical servers under control of a vCenter management server need to be licensed. To understand the impact and possible solutions to this problem, we need to know where this claim originates from. The source is what is stated in various versions of Oracle’s licensing guides and – as license guides on the web have no legal power – most likely also to be found in the contract that customers sign with Oracle. These documents say:
“All processors where the database is installed and/or running must be licensed.”
As Oracle does not recognise virtual machines, they require every physical host to be fully licensed regardless of how much virtual CPUs you use. So let’s say we have a VMware cluster with 4 hosts that run VMs with Oracle database. There’s another cluster with 8 hosts that run stuff like applications, web services, etc. The 4-node DB cluster is fully licensed but the 8-node App cluster obviously is not – because we don’t intend to run Oracle software on there, ever. With vSphere 5.1, a VM running Oracle on one of the 4 licensed nodes, could VMotion to one of the 8 unlicensed nodes in the App cluster. Even if a VMotion move would never happen (due to VMware DRS policies), Oracle will claim that the database (software) is installed because you could vmotion if you wanted. The fact that you never actually will doesn’t impress them.
This led to the funny blog post on House of Brick: The Oracle Parking Garage (still love that one).
Oracle is telling customers everywhere (without going in details) that you need to license all of your VMware ESX hosts (some customers were told that all ESX hosts in the datacenter would need to be licensed if they would run Oracle, even on only one host – without making clear that this would only be the case if everything was under one Vcenter server’s control). FUD – but sort of understandable given that VMware has a huge license savings potential. So what will happen in the future? Just some ridiculous speculation:
In Vsphere 9, Vmware will allow Vmotion to any vSphere servers connected to the internet. Therefore, from that moment onwards, Oracle will require 500 million processor licenses for every host running VMs with Oracle. Because you could not just have parked your car at any spot in this garage but you could have parked your car in any garage in the world.
So what can we do to cage the VMotion license dragon? The key is in the very wording that Oracle’s statements are based upon:
All processors where the database [software] is installed need to be licensed.
So all we have to do is make sure the software is NEVER installed on non-licensed servers. Now how can we make sure that:
- VMotion to non-licensed servers will NEVER take place
- Oracle software will NEVER even be visible on non-licensed hosts
A few weeks ago I was in Switzerland to discuss this very topic with one of our customers (I’m not giving names – but I want to thank them for the interesting brainstorm discussion we had and I will share the results here). As the risk of not being compliant is high, I targeted for having more than one layer of protection for both 1 & 2. Let’s start with the VMotion stuff.
VMotion checks a few things before actually attempting a VM move to make sure the VM can run accessing the same external resources as before the move. This includes access to storage (virtual disks) as well as networks (VLANs). Another thing VMotion requires is a network connection between the VMkernels of both ESX hosts. So if we want to block illegal VMotion moves we can do the following:
- Create a VLAN (using external IP backbone switches for example) and ONLY provide access to this VLAN to licensed hosts. Configure virtual machines running Oracle to make use of that VLAN – many organizations already have a separate “Server” VLAN anyway as well as “Admin”, “Backup” VLANs. Creating a dedicated “Oracle networking” VLAN should be trivial. Vmotion will not allow a VM to move if the target server cannot provide access to this VLAN.
- Isolate the VMkernels (using physical disconnection, firewall, etc) between the licensed and the non-licensed servers such that VMotion cannot work.
- Isolate the datastores holding volumes for database VMs to be only visible on licensed hosts.
We now have three roadblocks to prevent running on illegal hosts. What about software being available? The database software is usually installed either on the boot volume (typically a “VMDK” file – appearing on the virtual machine as /dev/sda) or on a separate volume (/dev/sdb). Put the VMDK files for these virtual (boot or software) volumes in a dedicated datastore. Make sure that this datastore is completely invisible on non-licensed hosts:
- Use SAN masking to isolate the software datastore to licensed hosts only
- Use SAN zoning to do the same
- Use dedicated physical SAN ports (Storage, SAN switches and hosts depending on how far you want to go) so that Oracle software is physically isolated
We now have 3 roadblocks to prevent software being “installed” on illegal hosts.
Let’s take a look at what it would look like:
You can see how we isolated VLAN traffic (there’s a routing connection so that applications can still connect to the databases, just not on the same VLAN), isolated VMkernel traffic and isolated datastore access to the binaries.
Also we implemented dedicated storage zones for boot volumes using zoning and masking (i.e. holding Oracle DB software) and isolated SAN connections so that non-licensed hosts can’t see the database software.
Finally, you should keep VMotion audit logs and store them on a location where they are out of reach for accidental deletes or tampering. Never share these logs with Oracle as they have no contractual rights to get these files and things can only get more complicated (in the disadvantage of the customer) if they get them.
What crazy claim will Oracle come up next with?
This post first appeared on Dirty Cache by Bart Sjerps. Copyright © 2011 – 2016. All rights reserved. Not to be reproduced for commercial purposes without written permission.
Hi Bart,
very good article, as usual..
My personal comment on this is that this might be due to the fact the OVM, as of today, lacks a lot of features that we are used to play with on VMware since years. Just few days ago a customer was complaining because of a migration of his OVM environment, from one storage to another, that will cause a huge downtime due to the fact that OVM doesn’t support a “storage vMotion” like functionality. I think that oracle will change this stupid licensing model as soon as their Virtualization solution will be able (will it?!?!?…) to support such features.
Stefano.
Have a look here: http://www.vmware.com/uk/business-critical-apps/oracle-virtualization/index
Has the standard Oracle contract been updated to reflect these assertions or are they anecdotal?
‘Oracle now claims that all physical servers under control of a vCenter management server need to be licensed.’ ‘Oracle is telling customers everywhere (without going in details) that you need to license all of your VMware ESX hosts’
The only terms that count are the ones in the agreement the customer and Oracle has signed.
Hi Terry,
I have no insight in what Oracle puts in their contracts. I only respond to the concerns of my customers in this case. So yes, anecdotal. Do your customers raise the same issues with you?
Thanks for commenting!
Hi Bart, Thanks for sharing your experiences … very interesting.
On my side, as i’m a bit paranoiaque, I would add to your caging list :
– Add a dedicated vCenter for Oracle hosts
– For VMWare 6.x , license Oracle Esxi hosts only with “vsphere standard” which does not include the cross vCenter vmotion
Hi Frank,
In this case I guess being paranoid is a good thing 🙂
Dedicated vCenter: That may help and be a good thing but some people like to have one point of control. And with vSphere 6 you can vMotion across multiple vCenters anyway so I’m not sure how this adds another barrier. Any insights on that?
vSphere Standard: correct – it does not allow the cross vcenter vmotion. But it also lacks DRS for example – which I think is a fundamental component in balancing varying workloads across multiphe physical hosts. Although it might solve the vmotion issue it may also cripple your architecture a bit. I guess it would be a good idea for smaller shops that would have only one or two hosts licensed for Oracle anyway. Any bigger and I think DRS is a must.
Thanks for the interesting thoughts!
IMHO the phrase “All processors where the database is installed need to be licensed.” makes absolutely no sense at all to start with.
Further, I guess they can always argue that you put all those measures in place only for their audit.
Last, I also guess they probably won’t name or create or accept any tool that would enable you to prove that you never performed any vMotion beyond what you claim.
So, they can do whatever they please. And I hope everyone fights back.
Hi Marki,
I agree, as you cannot install database software on a processor. You can arguably install it on a server or maybe disk. Also the term ‘processor’ needs to be defined as it can mean various things. But this is what’s in the contract so that’s what we have to work with.
Also agree on that we put those measures in place to avoid audit problems. Oracle may accuse customers of that, and even be right, but those measures prevent customers become out of compliance. How can Oracle be against anything that prevents their software to be used without license fees? 😉
And sure, Oracle can do whatever they please and not accept our tools BUT – and this is what everyone seems to forget – Oracle is NOT above the law.
Say the police is accusing me of a crime, but I can prove, using some kind of tool, with 100% certainty that I was not at the place of the crime when it was committed (alibi), can the police just ignore that because they don’t like the tool? Well maybe they can but I will not be convicted (in a country with a functional legal system at least).
IMO the problem is not so much the technology or contracts or methods but customers being threatened with legal actions that will never really happen (try to find cases where customers lost such cases against Oracle – there aren’t any AFAIK).
So far the chance of being hit by a meteorite is bigger than being convicted in court for running Oracle on VMware 😉