Programming Guide

This section provides the details of the programming requirements to operate the Ethernet FMC hardware and customise functionality.

PHY registers #

For correct operation of the Ethernet FMC, the 4x Marvell Gigabit Ethernet PHYs must be properly configured over the MDIO bus (for more information, see PHY Configuration). The Marvell PHYs have registers that control the operation and functionality of the device and the Ethernet link. The registers have default values that are applied when the PHY is powered up and released from reset. The default register values are suitable for a wide range of use cases, but some of them you might want to change to suit your application.

The tables below list some of the registers and settings that are useful for basic operation of the Ethernet FMC. For a comprehensive list of the registers, please refer to the Marvell Marvell 88E151x public datasheet .

Copper Control Register, Copper Control Reg, Page 0, Reg 0 #

Copper Status Register, Copper Status Reg, Page 0, Reg 1 #

Copper Auto-Negotiation Advertisement Register, Copper Auto-Neg Advertisement Reg, Page 0, Reg 4 #

1000BASE-T Control Register, 1000BASE-T Control Reg, Page 0, Reg 9 #

Copper Specific Status Register 1, Copper Specific Status Reg 1, Page 0, Reg 17 #

Copper Interrupt Status Register, Copper Status Reg, Page 0, Reg 19 #

Page Address, Page Address Any Page, Reg 22 #

When accessing registers, we must set the page address register to the value of the page that is being targetted. For example, when accessing the MAC Specific Control Reg 2 (MAC Specific Control Reg 2, Page 2, Reg 21), we must first set the page address register (register 22) to 2 prior to performing the register access.

MAC Specific Control Reg 2, MAC Specific Control Reg 2, Page 2, Reg 21 #

Autonegotiation #

After a hardware reset, the PHYs are configured by default to auto-negotiate with the link partner and advertise all possible link capabilities (1Gbps, 100Mbps, 10Mbps, full and half duplex). If desired, we can setup and initiate the auto-negotiation process manually through the PHY registers. One suggested method for doing this is illustrated by the following pseudocode:

1. Set Page Address Any Page, Reg 22 to 0

2. Read value of Copper Auto-Neg Advertisement Reg, Page 0, Reg 4

   • Enable asymmetric pause (assert bit 11)
   • Enable MAC pause (assert bit 10)
   • Advertise 100Mbps, 10Mbps, full and half duplex (assert bits 8,7,6 and 5)

3. Write new value to Copper Auto-Neg Advertisement Reg, Page 0, Reg 4

4. Read value of 1000BASE-T Control Reg, Page 0, Reg 9

   • Advertise 1Gbps full and half duplex (assert bits 9 and 8)

5. Write new value to 1000BASE-T Control Reg, Page 0, Reg 9

6. Read value of Copper Control Reg, Page 0, Reg 0

   • Enable auto-negotiation (assert bit 12)
   • Restart auto-negotiation process (assert bit 9)

7. Write new value to Copper Control Reg, Page 0, Reg 0

8. Read value of Copper Control Reg, Page 0, Reg 0

   • Trigger software reset (assert bit 15)

9. Write new value to Copper Control Reg, Page 0, Reg 0

10. Check the value of Copper Control Reg, Page 0, Reg 0 until software reset (bit 15) is deasserted

11. Read value of Copper Status Reg, Page 0, Reg 1

12. If auto-negotiation NOT complete (bit 5 deasserted) then

• Wait for 1 second
• Read value of Copper Interrupt Status Reg, Page 0, Reg 19
• If auto-negotiation error (bit 15), then abort the process
• Read value of Copper Status Reg, Page 0, Reg 1
• Repeat this process

13. Read value of Copper Specific Status Reg 1, Page 0, Reg 17 to determine link speed from bits 15:14

RGMII timing #

As described in RGMII Interface Timing Considerations, the RGMII RX and TX clock skews must be appropriately configured in the PHY for proper operation of the Ethernet FMC. By default, both the RX and TX clock skews are enabled in the PHY, however when using the AMD Xilinx AXI Ethernet Subsystem IP , the TX clock skew must be disabled. The following pseudocode illustrates how to disable the TX clock skew, while leaving the RX clock skew enabled:

1. Set Page Address Any Page, Reg 22 to 2

2. Read value of MAC Specific Control Reg 2, Page 2, Reg 21

   • Disable TX clock delay (deassert bit 4)
   • Enable RX clock delay (assert bit 5)

3. Write new value to MAC Specific Control Reg 2, Page 2, Reg 21

Note that when using the PS GEM with the GMII-to-RGMII converter IP, both TX and RX clock delays must be enabled in the PHY (the default configuration). Ensure however that the “TX clock skew added by PHY” option is enabled in that IP.

Processorless design #

If you wish to use the Ethernet FMC in a processorless design (ie. without a processor to setup the PHYs or perform packet handling), then we suggest you start with our processorless example design:

• Processorless design source code
• Processorless design documentation

You can also read more about the topic in this three part blog:

1. Driving Ethernet Ports without a processor
2. Processorless Ethernet: Part 2
3. Processorless Ethernet: Part 3

EEPROM #

The 2K EEPROM is intended to store information that identifies the mezzanine card and also specifies the power supplies required by the card. This information is typically read by the system power management on the carrier board when it is powered up. In typical user applications, it should not be necessary to read the data on the EEPROM, and we highly recommend against writing to the EEPROM. Nevertheless, if you wish to access the EEPROM, it can be read and written to at the I2C address 0x50.

The EEPROM sits on the FMC card’s dedicated I2C bus. The FMC pins of the EEPROM’s I2C bus are shown below, and it is up to the user to determine their corresponding connections to the FPGA/MPSoC on the carrier board being used.

FMC EEPROM Tool #

The Opsero FMC EEPROM Tool can be used to verify, reprogram or update the EEPROM contents of Opsero FMC products using an FPGA or MPSoC board such as the ZCU102 or VCU118 board.

Supported boards #

The tool currently supports the following FPGA/MPSoC boards. You must have at least one of these boards in order to use the tool.

• KC705
• KCU105
• VCU118
• VCK190
• VMK180
• ZedBoard
• ZCU102 Rev1.0 and Rev1.1
• ZCU104
• ZCU106

Download #

The tool can be downloaded at the link below:

Opsero FMC EEPROM Tool v1.5

The zip file contains a boot file (bitstream or BOOT.bin) for each of the supported boards.

Usage instructions #

To run the tool, follow these steps:

1. Plug the FMC card you wish to reprogram into one of the FMC connectors of your FPGA/MPSoC board. The tool is designed to probe all of the FMC connectors on the FPGA/MPSoC board.

2. If you are using the ZedBoard, be sure to set the VADJ jumper setting to 1.8V. If you are using the KC705, be sure that your FMC card can support a VADJ of 2.5V, which is the default setting of that board.

3. Connect the UART of your FPGA/MPSoC board to a PC.

4. For Zynq and Zynq MP boards, a BOOT.bin file is provided. Copy this file to your board’s SD card and configure it to boot from SD card. Then plug the SD card back into the board and power it up.

5. For FPGA boards, a bitstream is provided with an embedded ELF file. Power up your FPGA/MPSoC board and then download the bitstream to the FPGA board using the Vivado Hardware Manager tool.

6. Open a terminal program such as Putty and connect to the serial port of your FPGA/MPSoC board. If you see nothing in the terminal window, press ENTER to redisplay the menu.

7. Use the menu options to do the following:

   • Program the EEPROM (p)
     You will be asked to select the FMC product from a list, and also to enter the product’s serial number. Note that entering incorrect information here can lead to your FMC card being damaged by a VADJ voltage that is greater than it’s true rating. If you are not sure about the product to select here, please contact Opsero first.

   • Add a spacer (a)
     For 2-byte address EEPROMs only. This option allows you to add a spacer byte before the IPMI FRU data on the EEPROM. This can be used to help some FPGA/MPSoC boards to properly read 2-byte address EEPROMs. See below for an explanation.

   • Remove the spacer (r)
     For 2-byte address EEPROMs only. This option allows you to remove a spacer byte from the EEPROM.

Use of the spacer #

As referred to in this document and by the tool, a “spacer” is simply a single byte that is inserted into the EEPROM at address 0x00 and preceeding the IPMI FRU data.

In order to understand the purpose of the spacer, it is necessary to understand the two types of I2C EEPROM: 1-byte address and 2-byte address EEPROMs (also known in industry as “standard” and “smart serial” EEPROMs). The difference between these EEPROMs comes down to how many bytes are sent by the I2C master to specify the address from which it wants to read, or to which it wants to write.

After locating an EEPROM, the first thing that this tool does is determine if the EEPROM uses an address field of 1-byte or 2-bytes. This information is displayed in the EEPROM status of the main menu.

The VITA 57.1 standard specifies that FMC cards should have an EEPROM with an address field of 1-byte. FPGA/MPSoC boards are thus designed to communicate with a 1-byte address EEPROM on the FMC card. Communication issues can arise when using FMC cards that deviate from the standard and have a 2-byte address EEPROM. In this configuration, the FPGA/MPSoC board will send I2C commands with a 1-byte address field, but the EEPROM is expecting a 2-byte address field.

At first glance, this incompatibility seems like a big problem, but in fact we can work around it because the master’s interaction with the EEPROM is minimal. The FPGA/MPSoC board only needs to read from the FMC’s EEPROM, not write to it. When only reading from an I2C EEPROM, it actually doesn’t matter if the EEPROM has a 1-byte or a 2-byte address field. This is because the read command doesn’t have an address field. Instead, a read command returns data starting from the address pointed to by the internal address register in the EEPROM. As each byte is read from the EEPROM, the internal address register increments by one.

Typically, the I2C master would prepare for reading an EEPROM by first setting the internal address register to the address that it wants to read from. To set the internal address register, the master sends a write command containing an address field set to the desired value. If the master device only sends one byte for the address, a 2-byte address EEPROM would ignore the write request, considering it to be incomplete. Luckily in our case, this is inconsequential, because the IPMI FRU data starts from address ZERO, and the internal address register defaults to ZERO on power-up.

So why is there even a problem? Well, it turns out that some FPGA/MPSoC boards start off by sending a read command, and reading just one byte. This is probably done just to make sure that an EEPROM is actually connected. This initial single byte read causes the internal address register to be incremented to ONE. After the initial read, the master then tries to set the internal address register to ZERO and proceeds to read the entire EEPROM. As explained earlier, the master will fail at setting the internal address register, so it will remain pointing at address ONE and the IPMI FRU data will be read out starting from the second byte (ie. it will be missing the first byte).

By adding a dummy byte (spacer) to the beginning of the EEPROM, we can ensure that the IPMI FRU data is correctly read by the master.

Note that not all FPGA/MPSoC boards are programmed to send an initial single-byte read command on start-up, so they don’t all need the spacer. The KCU105 board is one example of this.

When to use the spacer #

If your FMC card has a 1-byte address EEPROM, this tool will NOT give you the option to add or remove a spacer, as you will have no need for one. All FPGA/MPSoC boards should be able to read 1-byte address EEPROMs on the FMC card as per the VITA 57.1 standard.

If your FMC card has a 2-byte address EEPROM, this tool will give you the option to add or remove a spacer in your FMC’s EEPROM. Adding a spacer can allow some FPGA/MPSoC boards to properly read a 2-byte address EEPROM.

A spacer has been found to work with the following boards:

• ZCU102
• ZCU102-ES2
• VCU118

The following boards will work without the spacer:

• KCU105

If your FPGA/MPSoC board is not listed above and is not applying VADJ to your FMC card, then try adding or removing the spacer. If your board is not applying VADJ regardless of the spacer being there or not, please contact Opsero .