I started writing this post in early January this year and before it gets completely old and (maybe) uninteresting, I hereby just post it, wondering what you might think. Of course I wanted to be it the ultimate perfect braindump but I realize that I can’t get my head around all the issues anyway and I am actually only interested if it has some merit or food for thought quality 😎
Something in the database performance field is bugging me for years, how do you compare computer systems performance wise. I am a guy with a holistic approach to performance when it comes to problems, so even if the performance consultancy request, is pinpointed on database level, I always take some steps back and look at, try to look at the overall issue: “A task, an amount of work, is not serviced in an expected time period”. So which part is not efficient enough, which part does not work as expected?
So lets take a step back and think it over.
The idea
My old colleague Gerwin Hendriksen and friend, and I tried to find the “X factor” combining all performance indicators across a complete architecture. Gerwin’s GAAP Method is a method that tries to address this issue (See the “General Approach Performance Profiling†whitepaper).
“Method-R†(Millsap 2003) describes how end-user processes are influenced by the “chain†of tiers in a technical infrastructure. In figure 1 is shown how different end-user processes response times (R) are influenced by different tiers of a technical infrastructure. Important to realize is the fact that the response time (R) of each individual tier is determined by the wait / queuing time (Q) and the service time (S), so R=Q+S. Further that the end-user process response time (R) is basically the sum of all the individual tier response times, so: R=RAS+RNET+RDB+RSAN+RSTOR.
For example the purple process on the application tier, shows some queuing time (Q) and service time (S) only on the application server, but for the red process all tiers are used. So tunings effort on the database server will not have any effect on the purple process, but will have influence on the red process, etc.
Sequence diagram to make clear how Cary Millsap’s Method-R (Millsap 2003) is describing how different processes (different colours), are using the architecture in different ways. In the picture are five tiers mentioned: Application Server (AS), Network (NET), Database Server (DS), Storage Area Network (SAN) and Storage (STOR). For the network (NET) and the storage area network (SAN) is only a line drawn, to make the diagram simpler. The blocks on the left side of the line is queuing time (Q) and on the right side of the line service time (S), for each tier.
Same but different approach?
My line of thinking is the following. I want to be able to compare the whole performance “track”, from the first user enter key to the actual data in the database, via middle tier processes to get there, and back. So to be able to not only compare systems, but also on a high detail level, I would need a common denominator to be able to show a “graph across the system”. That is from the first “enter” signal and back, ” response” (on a screen). If you think about it aggressively, there is always, I think one central factor: Energy.
Especially in our world today, energy is a dominant factor and thinking about it in a computer architectural sense, even when “doing nothing”, we are using energy, for example, a process is “waiting” for CPU cycles. The “waiting” itself will consume electricity, energy.
So overall, for example in a screen, show me result or action on input X; you should be able to determine and more interestingly compare a system, architecture or maybe framework, Y, in terms of energy consumption. So if I, Oracle, or any other vendor, would be able to compare “facts” and not just tweaked performance units, it must be one central based entity. Therefore I came up with this “Performance Efficiency”. In my mind “Performance Efficiency” is what the user is actually experiencing. Its this general “X” factor that binds it all together.
So in other words, if given X Watt of energy, how much data can be shown in process Y on screen via a predefined architectural setup. Even more, instead of the unit “costs” in an Oracle database, replace it with, energy per time using spent on retrieving X data… I called it, line of reasoning, Performance Energy Efficiency (PEE), or Performance Efficiency for short (PE)…so Watt/s (electrical energy unit per time unit).
In an computer environment/architecture –
Performance Efficiency
The amount of defined work that can be done with a certain energy amount in a certain amount of time.
Using the described Method-R method as a basis for Performance Efficiency (PE) where Response (R) time is described as Queue time (Q) plus Service time (S), Performance Efficiency then is the amount of Energy spend for the needed Response time: PE = E / R or PE = E / (Q + S) (in Watt/s).
Practical sense?
The practical advantage to describe it in that way is that for a certain amount of work/task, we now can compare this via the same unit, work done per time unit (at least in my mind). let me try to explain. The workload as “get me the first 10 records of a table”, could now be compared between two or more different database software suppliers, while dealing with the exact same environmental denominators like operating system, hard disks, CPU amount, etc. So in an Oracle environment, instead of an explain plan output saying “costs”, each part could now be described in energy unit needed per time.
Or, for example dealing with different virtual machine environments, lets say VMware contra XEN, comparing the exact same virtual machine setups, which one is more performance (energy) efficient? I saw a while a go a whitepaper describing energy efficiency trying to simulate the same setups but with different ZFS, EXT3, etc setups which looked rather what I have in mind pinpointing on IO operations per second (IOPS). Of course, I can’t find that whitepaper anymore, which is a shame because it was very interesting…
Instrumentation
As a peer via a email discussion mentioned: “It sounds like an interesting idea, but how would you measure it?”. To be honest, I tried to reason the implications of the “measure performance based on energy” idea, and got completely stuck in all the tech stuff that would be needed. On the other hand, I only want to address the idea in this post, just so its out there and not keep it as a “draft” somewhere internally. One of the great things of the internet that it is also an big pool of idea’s people can embrace or reject, but at least they are out there and might sparkle some other idea(s) that might be worth following up.
Hopefully the idea described here triggers something positive.
M.
