|
Hytec Electronics Ltd |
| Hytec Home Page | Writing Hotswap EPICS drivers for Industry Pack (IP) card |
 |
1) Introduction |
|
Walter
Scott (Scotty) |
|
2) Basic concepts and restrictions Only the software issues are discussed in this document. It is assumed that the hardware is hotswappable, i.e. it adheres to the VME64X standard (at least for the most part, see section 3 for slight deviations from the standard that can be accomodated in software). Note that only the carrier card, but not the IP card hardware, need conform to the standard. If the IP driver software is correctly written, any IP card will be hotswappable if it is mounted on a hotswappable carrier card such as the Hytec 8002 carrier card. In general, if boards can be removed from and inserted into a running system at any time, this poses all sorts of problems for the software. In this document, we restrict ourselves to the case where we want to replace a faulty piece of equipment in a running system. We therefore assume that: Every VME slot must have a carrier board with functionally the same hardware before and after the swap. This restriction allows the following swaps: a) the replacement of a faulty IP card with a functional equivalent. b) the replacement of a faulty carrier card. c) multiple carrier cards can be removed from the system at any given point in time. e.g. it is legitimate to remove two or more carrier cards, replace any number of IP cards with functional equivalents on the removed carriers, and then to replace the carriers into the identical VME slots. This restriction disallows the following swaps: a) moving of a carrier card from a VME slot n to m. b) adding an IP card to an empty IP slot on a carrier. c) adding a new VME64x carrier card to the system. If any of these operations need to be performed, then the system will need to be reconfigured, which typically requires at least a reboot of the IOC. 3) Software Design 3.1 Overview There are two essential elements to supporting hotswap in software. First, a removal and reinsertion event has to be recognised. Secondly, the software must know how to react when these events occur. ----------- ---------- ----- carrier status flag XXX YYY ----------- actual board presence (1=board removed) ------- -------- ^ ^ | | removal reinsertion Fig 1.: approximate time-state diagram of a carrier card removal and reinsertion event. There is a time delay when using polling to determine removal and reinsertion events, marked as XXX for removal and YYY for reinsertion. The delay marked XXX can affect system integrity if a driver attempts to access a carrier after it has been removed, but before this has been detected by the carrier polling task. 3.2 Detecting Card Removal There are two techniques for detecting the removal of a carrier card. 3.2.1 Interrupt According to the VME64X draft specifications, a carrier should generate an interrupt when the card is withdrawn. When the handles of the carrier card have been flipped up, but before the card can be removed completely from the crate, the card will generate an interrupt. The VME64X draft specifications furthermore state that the VME slot number should be put on the bus by the card. In this way, the processor can quickly determine when which card is being removed. Note that the VME64X specifications are a draft standard, and are likely to change in the future. Often IP cards on the carrier will generate their own interrupts for their own functionality. It is necessary for the carrier to have its own interrupt vector to handle the removal event. It is advisable to choose a higher priority for the card removal interrupt level than for the IP interrupt levels. In this scenario, the card removal interrupt service routine (ISR) sets a flag for the presence of each carrier card. The IP driver routines check the status of this flag before accessing the hardware in the usual way. Note that this does not guarantee trouble free operation when removing a carrier card because the checking of the carrier status flag and hardware register access cannot be performed as an atomic operation. For example, a bus error can occur in the following case: The IP driver checks the carrier present flag and finds the card to be present (flag=true). Right after this, the card is removed, generating an interrupt. As a consequence, the high priority interrupt handler sets the carrier flag to false. Once the interrupt handler exits, the IP driver resumes running. If it now accesses the nonexistant hardware registers in the normal way (i.e. without using vxMemProbe()), it will terminate with a bus error. While this scenario will be very rare, it cannot be fully excluded. 3.2.1 Poll and Probe If a carrier card does not generate an interrupt on its removal, this event can nevertheless be recognised by periodic polling. This method can be made to work reliably if: a) the polling task runs at a higher priority than any other task accessing the hardware to be hotswapped. b) the driver routine accesses the hardware by using vxMemProbe(). Condition a) ensures that the data flags signalling the absence of a board is not accessed when the polling task is updating them. This is a mutex on the shared "presence flag" data based on task priority. Condition b) ensures that a bus error will not occur if the hardware is accessed after a card is removed just after periodic polling has found it to be present (see the delay marked with XXX in figure 1). 3.3 Detecting Card Insertion The VME64X specification requires reinsertion of a board to be detected by polling. This is best performed by a high priority task at the carrier level. The IP driver level must check the carrier status flag immediately before accessing hardware registers in order to avoid a bus error. The delay in the detection of a reinsertion event is not critical to the correct functioning of the system (see figure 1). 3.4 Acting on Card Removal and Reinsertion The IP driver routines should check for the existence of the hardware before accessing hardware registers. A copy of the hardware registers value should be kept in RAM for each hardware device. When the hardware has been removed, the routines should return a status value indicating the fact to the EPICS record level. Upon reinsertion, the driver should reinitialise the hardware registers from the previously saved values in RAM where this makes sense. 3.5 Discussion Note that, in principle, carrier card polling or interrupt handling is not necessary if the IP driver level uses vxMemprobe() for all hardware access. However, it is more efficient to handle removal and reinsertion events at the carrier board level rather than for each IP card. In addition, it is advantageous in terms of portability for the IP driver to be able to question the carrierfor its existence in the system, regardless of how board removal and reinsertion events are detected. For this reason, the drvIpac header file has been extended with a query function ipmCarrierIsPresent(). In addition, IP drivers should access hardware registers using the VxMemProbe() function to avoid bus errors. 4) Acknowledgements Steve Hunt is the man who first wrote hotswappable EPICS drivers. This document is the result of useful discussions with him. Of course, all errors are entirely mine. 5) Bibliography For much documentation about EPICS, including a list of all EPICS supported hardware, see the EPICS homepage at http://www.aps.anl.gov [1] The EPICS IPAC drvIpac Module was written by Andrew Johnson <anjohnson@iee.org>. The homepage of this software is http://www.aps.anl.gov/asd/people/anj/ipac [2] The Hytec Electronics 8002 VME64X industry pack carrier board hardware user manual can be downloaded from the Hytec web site: http://www.Hytec-electronics.co.uk [3] The Hytec VICB8002 drvIpac driver user manual can be downloaded from the Hytec web site: http://www.Hytec-electronics.co.uk |
For information on Hytec
products and services please contact: This site is best viewed
with Internet Explorer some problems may be experienced
with Netscape V6.01.
HYTEC Head Office
Post : 5 Cradock Road, Reading, Berkshire,
RG2 0JT, England.
Phone : +44 (0)118 9757770
Fax : +44 (0)118 9757566
LE3 2FB,
England.
Copyright © 2001 [Hytec Electronics Ltd].
All rights reserved.
Information in this document is subject to change without notice.
Other products and companies referred to herein are trademarks or registered
trademarks
of their respective companies or mark holders. send mail to paul@hytec-electronics.co.uk
with questions or comments about this web site.Last modified: July 27, 2005