Before you start
Objectives: learn what Fiber Channel is, when it should be used, and what components does it require.
Prerequisites: you should know what is PATA, SATA, SCSI, RAID, and about fiber optic cables.
Key terms: fiber channel, FC, fiber, optic cable, storage system
What is Fiber Channel (FC)
When we install storage devices in our computer, we have several technologies which we can use. For example, we can use PATA, SATA or SCSI standards to connect our hard disks to the motherboard. In all those technologies we actually use specific kind of copper wires to connect the hard drives to the motherboard. Those cables are used to carry electrical signals between devices.
All this works fine on normal, desktop computers, but what about server environments? In server environment, we should be concerned about the speed of data transfer on our hard disks. Ordinary hard disks in server environment will may be too slow for all our transfer needs. If our server is in high demand, our ordinary disks will slow the data transfer down, since they may be to slow for the high demand.
If our storage is too slow, we can step away from hard disks which use electrical signals and choose devices which use light impulses to transfer data. We call this technology the Fiber Channel (FC). Fiber Channel uses fiber optic cables, instead of copper wiring, to transfer data by using light impulses. FC is designed to provide a method for transferring large amounts of data, and transfer it fast. So, we use FC hardware to interconnect storage devices with servers.
How Fiber Channel Works
The great thing about fiber channel is that it is very fast. If we want to set up a Fiber Channel storage system, the first thing we have to do is make sure that we have a fiber channel adapter installed on our system. Some servers will come with that adapter already installed or integrated. In other cases we have to purchase the adapter separately. The next thing we have to implement is the storage itself. When working with fiber channel, we will usually connect to a externally configured RAID array or tape storage system. So, the storage is actually placed outside of the server system. The RAID array or Tape storage device will have a fiber channel interface built in, so we can connect the RAID array with our server using fiber optic cabling.
Fiber Optic Cabling
Fiber optic cables can come in two different modes. The first mode is called Multi mode, which uses thick plastic or glass core in the middle of the cable, from 50 to 100 microns in diameter. The benefit of Multi mode cable is that it can carry multiple rays of light (modes), at the same time concurrently. It does this by using the natural tendency of light to bounce inside of cable. By bouncing light in different angles, the cable can carry multiple rays of light at the same time. The disadvantage of the bouncing is that the light signal will get more unclear the farther it travels. If it gets to far, it gets so unclear that it is no longer usable. Because of that the multi mode cables can have shorter ranges.
The second mode of fiber optic cable is Single Mode. In this cable, instead of multiple bouncing rays of light, we only have one ray of light at the time in the cable. It uses a very thin core when compared to multi mode cable, about 10 microns in diameter. The cladding of the cable keeps the ray of light centered in the cable. Because of that it will support longer lengths and higher bandwidths. The down side is that it is more expensive.
Because we are dealing with fiber optic cables, the length of the cable to connect our storage can be very long compared to technologies like PATA, SATA or SCSI.
Fiber Channel Standards
Over the years Fiber Channel has been implemented using several standards. When talking about standards, we first have to see which type of cabling is used to connect to our fiber channel storage. If we use a multi mode cable, the length of the cable can be up to 500 meters. If we use a single mode cable, the length of the cable can be 10 kilometers.
Fiber Channel is standardized by the T11 Technical Commitee which is part of the ANSI organization. T11 specifies several different types of fiber channel implementations that we can choose from. The first one is called 1GFC, which stands for 1 Gigabit Fiber Channel. The total speed of 1GFC is 1.0625 Gbit/s. The second is 2GFC which allows speed of 2.125 Gbit/s. The third is 4GFC which allows 4.25 Gbit/s. We also have 8GFC (8.5 Gbit/s), 10GFC (10.5 Gbit/s), and 16GFC (14.025 Gbit/s). It is expected to achieve even greater speeds in the future.
Fiber Channel Topologies
There are also several implementations or topologies of Fiber Channel. The first implementation is called Point-to-Point Fiber Channel. In this case, we connect our system (typically a server) using a fiber optic cable to the storage device. This implementation is often called FC-P2P, and is the simplest topology in which two devices are connected directly to each other. The limitation of this topology is that only one host system can use fiber channel device at a time.
In case we want multiple systems to be able to access the same fiber channel device, we have to use some other topology. In this case we actually have two options. One is called FC-AL which stands for Fiber Channel – Arbitrated Loop. With FC-AL we actually create a token ring network. Using FC-AL we can connect up to 126 different devices in a ring. The disadvantage of FC-AL is that if one link fails, the whole fiber channel system won’t work any more.
Another option to enable multiple systems to access fiber channel device is FC-SW which stands for Fiber Channel Switched Fabric. In this case we have a network similar to an Ethernet network. We have a fiber channel switch on which we connect all fiber channel devices and our server systems. So, this way all devices are interconnected and share the media bandwidth. This is great for clustering purposes because we can configure two or more systems to use the same storage device as if they were one, single server. If one server goes down, the other one can continue to work using the same data available on the same storage device.
So we see that in FC-SW the Ethernet technologies are used to provide connectivity. We should also mention that FC uses SCSI command interface. In this article we talked about fiber optic, but with FC we can also use twisted pair copper wiring. It can utilize multiple protocols, including IP and SCSI. When compared to SCSI, the FC controller ID is pre-configured with a unique ID. Data frames can be 2148 bytes long.
Fiber Channel Hardware
When talking about FC hardware, we can distinguish physical layer devices, interconnect devices, and translation devices. On physical layer we have fiber optic or copper cables. Cables carry FC signal between transceiver pairs. Interconnect devices are switches and routers. A Gigabit Interface Converter (GBIC) is used to connect network adapters to fiber. Network switches use an optical transceiver called Small Form-factor Pluggable GBIC (SPF GBIC) to connect to the fiber. Translation devices are the intermediaries between FC protocols and upper layer protocols such as SCSI, FCP, FICOM, Ethernet and SONET. The most common translation device is the Host Bus Adapter (HBA) or FC Adapter. An HBA connects the FC network to the server’s host bus, which can be PCI or serial bus. A bridge device can be used to connect legacy SCSI or ESCON storage devices to the FC network. An adapter device can be used to connect FC to IP network such as Ethernet. A gateway device (router) can be used to interface to telecom networks such as ATM or SONET.
Fiber Channel Connections (Ports)
Different types of connections made in FC networks are called ports. Those are:
- Network port (N_port) – a node port used to connect a node to a FC switch in a Point-to-Point or switched fabric topology.
- Expansion port (E_port) – a switch port used to cascade FC switches together. When E_ports between two switches form a link, that link is referred to as an InterSwitch Link or ISL.
- Expansion port (EX_port) – the connection between a FC router and a FC switch. On the side of the switch it is a normal E_port, but on the side of the router it is an EX_port.
- Generic port (G_port) – a switch port that can operate as an E_port or F_port.
- Fabric port (F_port) – a switch port used to connect FC fabric to a node (not loop capable).
- Fabric Loop port (FL_port) – a switch port that connects loops and switches in a public loop for an arbitrated loop topology. A switch port may automatically become either an F_port or an FL_port depending on what is connected.
- Node Loop port (NL_port) – a node port which connects a node to both loops and switches in an arbitrated loop topology.
- Loop port (L_port) – connects a node to a FC loop. It is the loose term used for any arbitrated loop port, NL_port, or FL_port.
- Trunking Expansion port (TE_port) – a term used for multiple E_ports trunked together to create high bandwidth between switches.