How is High Speed Digital IO designed and implemented?

Because the communication in the 5G and IoT era has the characteristics of high speed, low latency, high bandwidth, etc., to realize the connection of 5G and IoT, a large number of base stations need to be established, which will also bring a lot of power, signal, optical fiber and other connectors and cables Therefore, many high-speed digital interfaces have gradually come into our eyes. At present, the most common ones are PCIE, SAS and SATA. Of course, there are XAUI protocol and Thunderbolt on 10G Ethernet. This is jointly developed by Intel and Apple. Common high-speed interfaces that combine DisplayPort ports together, etc. Today we are going to share how High Speed Digital IO itself is designed and realized! For specific PCIE, SAS and other ranking explanations, you can just ask Du Niang: PCIe, SAS and SATA, who will lead the storage interface

High Speed Digital is usually a serial interface. It can also be said that it is because it is serial that the speed becomes so high, because it turns the data on the internal parallel bus of the chip into serial transmission, such as the above The QSFP 40G mentioned is also achieved through 10G*4x, but why must this be done? The SI problem of high-speed digital signals on the PCB is very complicated and difficult to design. Why can't the parallel data inside the chip be sent directly? Let's get to know it together!

This has to start with Moore’s Law. When the chip scale continues to grow, there are no more pins on the chip package, so it is impossible to pull all the IOs to the package separately, and friends who are familiar with packaging know that. The more the package pins increase, the higher the package cost. In order to meet the market price/performance ratio, we must also meet such requirements. In fact, fewer pins are actually good for SI (PI). Imagine that dozens of parallel IOs are flipped at the same time. , It will bring a large SSO (simultaneous switching ouput), the stability of the power supply will be very poor, the whole board will also have a large EMI, when dozens of single-ended parallel IOs become a pair of serial The problem of SSO is greatly reduced when the signal is used. Usually, high-speed serial interfaces are differential input and output. The differential signal itself has opposite currents. For the IC on the drive side, the current on the power supply is stable! So now I understand that the high-speed interface actually exists mainly to reduce the cost, but at the same time, a problem that the high-speed interface must solve is the board-level synchronization problem. When the IO speed is very slow, the delay caused by the board wiring is almost It can be ignored, but when the IO speed is very fast, the delay caused by the board wiring will be a few cycles away.

The figure shows the same trace length, the high-speed IO has been flipped for several cycles, and the low-speed IO has only experienced a rising edge
So at this time we can no longer use the previous system-level synchronization method, because the entire board is synchronized with the same clk, but the distance between this clk and each IC is different, it is definitely impossible to synchronize, unless it is stable and completely symmetrical. The lines are completely equal in length, then I think this kind of board should not be sold at all!
Another synchronization method is to drive the IC to send clk and data to the recieve side at the same time, and then configure the same length to ensure that the clock and data are aligned. For example, our common ddr is in this way, but this The more troublesome method is that there will still be very strict equal length requirements to transmit speeds above Gbps, which is equivalent to limiting the flexibility of wiring, which requires the training process to be added to the ddr3 and ddr4 chips to achieve read and write balance , Leaving more margin for us to route. In addition, when the receiving end receives the clk from the driver, it has to match the clk with the master clock of its own chip. Assuming a 32-bit ddr data, every 8 bits with a clk, so that the receiving end will receive 4 different clk, and then move them to the clock domain of the chip itself, so that there are 5 different clock domains in the chip Up. The chip design is also more troublesome.
No, none of these programs are good enough. A really good program should be concise and effective. Since sending clk and data at the same time is not so flexible, can you just send data and then extract the clk information from the data at the receiving end? In this way, there is no need for data to be synchronized with clk, nor for clk to be moved in the clock domain!
The answer is yes. Every part of the high-speed digital IO serves this purpose: effectively parse out data and clk, and then move them to the internal differential bus. The chip area added by these modules is a little more than the package. The cost of ten pins is nothing compared to the cost. Do you think that the people who invented the high-speed serial interface are too perverted, and the things that can be solved with money must rely on wisdom, which makes it more difficult for us ordinary people to follow.

The transmitting end must first have a parallel-to-serial structure, which can be realized by using a series D flip-flop. The data from the parallel bus is moved to the series D flip-flop one by one using the parallel clk as the enable signal. go with. Then move the phase of the internal parallel clk (for example, 90 degrees) to create four clocks of 0 degrees, 90 degrees, 180 degrees, and 270 degrees. When they are combined, the clock becomes 4 times the original one. Use this Clock, store the data in the D flip-flop before sending! The receiving end is also in a similar way, but the reverse is serial and parallel. The clock recovery in the data is done through pll. In fact, not only the clock is recovered here, but the control signal also needs to be separated from the data!
The PLL recovers according to the fastest frequency of the signal received on the Rx pin, and then compares with the Rx to become the internal Rx data!

Now that the data and clock are available, it is time for the so-called protocols to come into play. In the protocol, we define some combinations of databits as flags. When the receiving end receives these flags, it determines whether the next data is data, control, or idle!
One more sentence here, why should idle be distinguished? Wouldn’t it be over when sending a string of 0s? In fact, this is not the case. In order to return clk normally, try to ensure that the output data does not have a long string of 0 and 1. This involves our most common 8b/10b encoding, 64b/66b scrambling, etc., which are also defined in the agreement.