Transfer: https://blog.csdn.net/BaiWfg2/article/details/52885287

  Storage for DBAs: Ever been to one of those sushi restaurants where the food comes round in dishes on a conveyor belt? As each dish travels around the loop you eye it up and, as long as you can make your mind up in time, grab it. However, if you are as indecisive as me, there’s a chance it will be out of range before you come to your senses – in which case you have to wait for it to complete a further full revolution before getting another chance. And that’s assuming someone else doesn’t get to it first.

Have you ever been to a sushi restaurant where the food is placed on a conveyor belt? As each food item is conveyed on the belt, you target some food items and grab them as soon as they come to you. However, if you are as hesitant as I was, chances are the food is beyond your reach. At this time, you have to wait another circle before you can get it, provided that no one else has taken it.

  Let’s assume that it takes a dish exactly 4 minutes to complete a whole lap of the conveyor belt. And just for simplicity’s sake let’s also assume that no two dishes on the belt are identical. As a hungry diner you look in the little menu and see a particular dish which you decide you want. It’s somewhere on the belt, so how long will it take to arrive?

 Let’s assume that it takes 4 minutes for a food item to complete one revolution on the conveyor belt. For simplicity, we also assume that the foods on the conveyor belt are different from each other. As a foodie, you look at the menu and find a few items of food you want. It is somewhere on your belt. So how long will it take for it to arrive next to you?

  Probability dictates that it could be anywhere on the belt. It could be passing by right now, requiring no wait time – or it could have just passed out of reach, thus requiring 4 minutes of wait time to go all the way round again. As you follow this random method (choose from the menu then look at the belt) it makes sense that the average wait time will tend towards halfway between the min and max wait times, i.e. 2 minutes in this case. So every time you pick a dish you wait an average of 2 minutes: if you have eight dishes the odds say that you will spend (8 x 2) = 16 minutes waiting for your food. Welcome to the disk data diet, I hope you weren’t too hungry?

 We specify that it can be anywhere on the belt. It may be passing by and you can get it without waiting, or it may have just passed your range, so it will take 4 minutes to wait to complete a circle. When you follow this random set of rules (select food from the menu and then watch the conveyor belt), you will realize that the average wait time will tend to be somewhere in the middle of the maximum and minimum wait times, which is 2 minutes. So you have to wait 2 minutes every time you get your food. If you have 8 plates, you probably have to wait 16 minutes to get it. Welcome to Disk Data Diet, hope you’re not too hungry?

  Now let’s consider an alternative option, where you order eight dishes from the chef and he or she places all of them sequentially (i.e. next to each other) somewhere on the conveyor belt. That location is random, so again you might have to wait anywhere between 0 and 4 minutes (an average of 2 minutes) for the first dish to pass… but the next seven will follow one after the other with no wait time. So now, in this scenario, you only had to wait 2 minutes for all eight dishes. Much better.

 Now let’s consider another scenario. You order 8 dishes, and the chef places them somewhere on the conveyor belt in turn. The positions are random, so you need to wait an average of 2 minutes to get the first dish. However, there is no need to wait for the remaining 7 dishes. So in this scenario, you only need to wait 2 minutes to pick up 8 dishes, which is much better than before.

I’m sure you will have seen through my analogy right from the start. The conveyor belt is a hard disk and the sushi dishes are blocks which are being eaten / read. I haven’t yet worked out how to factor a bottle Asahi Super Dry into this story, but I’ll have one all the same thanks.

I’m sure you can understand the analogy I made at the beginning of this article. The conveyor belt is the disk and the food is like the block to be eaten/read. [The last sentence cannot be translated. 555……]

Random versus Sequential I/O   I have another article planned for later in this series which describes the inescapable mechanics of disk. For now though, I’ll outline the basics: every time you need to access a block on a disk drive, the disk actuator arm has to move the head to the correct track (the seek time), then the disk platter has to rotate to locate the correct sector (the rotational latency). This mechanical action takes time, just like the sushi travelling around the conveyor belt.

 I will have another article talking about the principle of disks in another day. But for now, let me outline the basics: every time a block on the disk is accessed, the magnetic arm needs to move to the correct track (this time is the seek time), and then the platter needs to rotate to the correct sector (this is called rotation delay). This sequence of actions takes time, just as it takes time for the sushi to move on the conveyor belt.

  Obviously the amount of time depends on where the head was previously located and how fortunate you are with the location of the sector on the platter: if it’s directly under the head you do not need to wait, but if it just passed the head you have to wait for a complete revolution. Even on the fastest 15k RPM disk that takes 4 milliseconds (15,000 rotations per minute = 250 rotations per second, which means one rotation is 1/250th of a second or 4ms). Admittedly that’s faster than the sushi in my earlier analogy, but the chances are you will need to read or write a far larger number of blocks than I can eat sushi dishes (and trust me, on a good day I can pack a fair few away).

 Obviously the total time depends on the initial position of the disk head, as well as the position of the sector to be accessed. If it is just under the head, there is no need to wait; if it has just passed the head, you have to wait for one cycle. Even for the fastest 15k RPM disk, 15,000 revolutions per minute and 250 revolutions per second, then one revolution takes 4ms. It’s obviously much faster than the sushi situation just now, but in many cases a large number of data blocks need to be read, which far exceeds the amount of sushi I want to eat. Believe me, I can pack several copies at a time like this.

  What about the next block? Well, if that next block is somewhere else on the disk, you will need to incur the same penalties of seek time and rotational latency. We call this type of operation a random I/O. But if the next block happened to be located directly after the previous one on the same track, the disk head would encounter it immediately afterwards, incurring no wait time (i.e. no latency). This, of course, is a sequential I/O.

What about the next disk block? If it is somewhere on the disk, accessing it will have the same seek and rotation delays. We call this method of IO random IO; but if it happens to be behind the disk block you just accessed, the head can encounter it immediately without waiting. This kind of IO is called sequential IO.

Size Matters   In my last post I described the Fundamental Characteristics of Storage: Latency, IOPS and Bandwidth (or Throughput). As a reminder, IOPS stands for I/Os Per Second and indicates the number of distinct Input/Output operations (i.e. reads or writes) that can take place within one second. You might use an IOPS figure to describe the amount of I/O created by a database, or you might use it when defining the maximum performance of a storage system. One is a real-world value and the other a theoretical maximum, but they both use the term IOPS.

 In my last blog post, I talked about the basic characteristics of disks: latency, IOPS and bandwidth (or throughput). Let me say it again, IOPS is the abbreviation of I/O per second, which represents the number of input and output operations (such as reading and writing) in one second. You can use the IOPS value to describe the amount of IO operations of a database, or use this term when defining the maximum performance of a storage system. The former is a real-world value, the latter is a theoretical maximum, and they both refer to the term IOPS.

  When describing volumes of data, things are slightly different. Bandwidth is usually used to describe the maximum theoretical limit of data transfer, while throughput is used to describe a real-world measurement. You might say that the bandwidth is the maximum possible throughput. Bandwidth and throughput figures are usually given in units of size over units of time, e.g. Mb/sec or GB/sec. It pays to look carefully at whether the unit is using bits (b) or bytes (B), otherwise you are likely to end up looking a bit silly (sadly, I speak from experience). In the previous post we stated that IOPS and throughput were related by the following relationship:

 The situation is different when describing large amounts of data. Bandwidth is used to describe the theoretical maximum value of data transmission, while throughput is the actual value. You could say that bandwidth is the upper limit of throughput. Bandwidth and throughput values ​​often have units of unit size per unit time, such as Mb/sec, Gb/sec. Note that b and B are different here, the former is bits and the latter is bytes. In the previous blog post, we talked about the relationship between IOPS and throughput:

1

Throughput   =   IOPS   x   I/O size

Throughput = IOPS * I/O size

  It’s time to start thinking about that I/O size now. If we read or write a single random block in one second then the number of IOPS is 1 and the I/O size is also 1 (I’m using a unit of “blocks” to keep things simple). The Throughput can therefore be calculated as (1 x 1) = 1 block / second.

 Now it is necessary to talk about IO size. If a single random block is read in one second, then the IOPS is 1 and the IO size is also 1 (blocks are used here to simplify the problem). Then the throughput is 1*1=1 block/s

  Alternatively, if we wanted to read or write eight contiguous blocks from disk as a sequential operation then this again would only result in the number of IOPS being 1, but this time the I/O size is 8. The throughput is therefore calculated as (1 x 8) = 8 blocks / second.   Hopefully you can see from this example the great benefit of sequential I/O on disk systems: it allows increased throughput. Every time you increase the I/O size you get a corresponding increase in throughput, while the IOPS figure remains resolutely fixed. But what happens if you increase the number of IOPS?

Or another way, read 8 consecutive data blocks sequentially, then the IOPS is still 1, but the size is 8, so the throughput is 1*8=8 blocks/s. I believe you can see the advantages of sequential IO. It supports incremental throughput. Each time the number of IO data blocks is increased, the throughput is improved, but the IOPS remains constant. What if it increases?

Latency Kills Disk Performance   In the example above I described a single-threaded process reading or writing a single random block on a disk. That I/O results in a certain amount of latency, as described earlier on (the seek time and rotational latency). We know that the average rotational latency of a 15k RPM disk is 4ms, so let’s add another millisecond for the disk head seek time and call the average I/O latency 5ms. How many (single-threaded) random IOPS can we perform if each operation incurs an average of 5ms wait? The answer is 1 second / 5 ms = 200 IOPS. Our process is hitting a physical limit of 200 IOPS on this disk.   What do you do if you need more IOPS? With a disk system you only really have one choice: add more disks. If each spindle can drive 200 IOPS and you require 80,000 IOPS then you need (80,000 / 200) = 400 spindles. Better clear some space in that data centre, eh?

In the example above, I described a single-threaded process reading and writing a single random block of disk. That kind of IO will have a large delay, such as the seek time and rotation delay mentioned above. It is already known that for a 15k RPM disk, the average rotation delay is 4ms. If we assume that the seek time of the disk head is 1ms, then the average IO delay is 5ms. In this case, each operation takes 5ms. How many operations can be performed in one second, which is the value of IOPS. The answer is 1s/5ms=200 IOPS (single-threaded). So what should you do if you want to increase IOPS? The only way is to add more disks. If one spindle drives 200 IOPS, then to reach a value of 80,000 IOPS, 80,000/200=400 spindles are needed. Is there more clarity on the space situation for this data center?

  On the other hand, if you can perform the I/O sequentially you may be able to reduce the IOPS requirement and increase the throughput, allowing the disk system to deliver more data. I know of Oracle customers who spend large amounts of time and resources carving up and re-ordering their data in order to allow queries to perform sequential I/O. They figure that the penalty incurred from all of this preparation is worth it in the long run, as subsequent queries perform better. That’s no surprise when the alternative was to add an extra wing to the data centre to house another bunch of disk arrays, plus more power and cooling to run them. This sort of “no pain, no gain” mentality used to be commonplace because there really weren’t any other options. Until now.

On the other hand, if you can perform sequential IO, you can reduce the IOPS requirements and increase throughput, allowing the disk system to transfer more data. I learned that Oracle users spend a lot of time and resources re-partitioning and sorting the data so that requests will be sequential IO. [My personal question here is: Can repartitioning and sorting the data ensure that the arrangement on the disk conforms to sequential IO? ] They believe that such initial preparation work, although troublesome, will be worth it in the long run because subsequent requests will be executed better. Of course, it is also possible to add additional disks to store more data, and to add more power and cooling devices. This NO PAIN, NO GAIN mentality is very common because there is no other choice so far

Flash Offers Another Way   The idea of sequential I/O doesn’t exist with flash memory, because there is no physical concept of blocks being adjacent or contiguous. Logically, two blocks may have consecutive block addresses, but this has no bearing on where the actual information is electronically stored. You might therefore say that all flash I/O is random, but in truth the principles of random I/O versus sequential I/O are disk concepts so don’t really apply. And since the latency of flash is sub-millisecond, it should be possible to see that, even for a single-threaded process, a much larger number of IOPS is possible. When we start considering concurrent operations things get even more interesting… but that topic is for another day.

  There is no concept of sequential IO in flash memory because there is no physical concept of contiguous or continuous blocks. Logically, two blocks can have consecutive block addresses, but there is no certainty where the actual information resides. You may say that flash IO is random, but in fact random IO and sequential IO are just disk concepts, so don’t use them like this. Since the latency of flash memory is sub-millisecond, it can have a large IOPS for a single-threaded process. It’s more interesting when concurrency is considered, but that’s for another time.

  Back to the sushi analogy, there is no longer a conveyor belt – the chefs are standing right in front of you. When you order a dish, it is placed in front of you immediately. Order a number of dishes and you might want to enlist the help of a few friends to eat in parallel, because the food will start arriving faster than you can eat it on your own. This is the world of flash memory, where hunger for data can be satisfied and appetites can be fulfilled. Time to break that disk diet, eh?

  Looking back at the disk model, all that sitting around waiting for the sushi conveyor belt just takes too long. Sure you can add more conveyor belts or try to get all of your sushi dishes arranged in a line, but at the end of the day the underlying problem remains: it’s disk. And now that there’s an alternative, disk just seems a bit too fishy to me…

 Going back to the sushi analogy, there is no conveyor belt anymore, the chef is standing right in front of you. When you order a food, it appears in front of you immediately; when ordering a lot of food, you can ask your friends to help you eat together, because the food will always be served faster than if you eat alone. This is the mechanism of flash memory implementation.

 Last paragraph, omitted