Slow Controls Upgrade Procedure

Updated Mar 2019 Jon Richardson

Hardware

Auxiliary DAQ Modules

The new auxiliary DAQ systems use the Acromag XT series DIN-rail mounted ADC, DAC and BIO modules. The model numbers are in the format XTYYY1, where "XT" is a static prefix, "YYY" determines the function, and the "1" as the last digit denotes the ModBus/TCP version of the model. The 1 at the end is important, as the other variations will not be able to interface with CDS through EPICS.

Auxiliary DAQ Chassis

Auxiliary DAQ Host Machine

The Acromag modules communicate with a central host computer on a local network. The computer runs an EPICS IOC which interfaces with the Acromag modules via Modbus/TCP and hosts the EPICS records for analog and binary auxiliary channels. For these systems we've chosen to use Supermicro SYS-5017A-EP 1U rackmount servers, which have two Ethernet interfaces. Having two network interfaces is critical for creating a closed local network between Acromags and host.

HowTo

Configure auxiliary DAQ host machine

The following has proven to work with Debian 8 (x64) on a Supermicro SYS-5017A-EP.

OS installation

1. Fresh install of Debian 8 (or most recent stable release)
   • Hostname: same as old system

   • Username: controls

   • Optional: lightweight desktop environment LXDE (recommended)

2. Give sudo privileges to user controls

   • $ su
     $ usermod -aG sudo controls
     $ exit

Networking

1. Assign the host machine an available IP address on the martian network, 192.168.113.XXX
   • XXX is an integer in the range 0-255. Do not use 0, 10, or 255 (reserved)

   • See here for the list of in-use addresses

2. Register the host machine in the DNS lookup table on chiara
   1. Connect to chiara

      $ ssh -X 192.168.113.104

   2. Add the following line to /var/lib/bind/martian.hosts

      HOSTNAME A 192.168.113.XXX

      • HOSTNAME should be replaced with the actual hostname

      • Search the file to make sure XXX is not already in use
   3. Add the following line to /var/lib/bind/rev.113.168.192.in-addr.arpa

      XXX PTR HOSTNAME

   4. Restart the DNS server and log out

      $ sudo service bind9 restart
      $ logout

3. Add lines "search martian" and "nameserver 192.168.113.104" to /etc/resolv.conf

4. Set up martian and local ethernet adapters
   • Edit /etc/network/interfaces as follows:

     allow-hotplug eth0
     iface eth0 inet static
             address 192.168.113.XX
             netmask 255.255.255.0
             gateway 192.168.113.2
             dns-nameservers 192.168.113.104 131.215.125.1 131.215.139.100
             dns-search martian
     
     allow-hotplug eth1
     iface eth1 inet static
             address 192.168.11Y.1
             netmask 255.255.255.0
             network 192.168.11Y.0
             broadcast 192.168.11Y.255

   • XX is a placeholder for the machine's IP on the martian network
   • Y is placeholder for the IP root of the local subnet

     Hostname	Martian IP	Local IP
     C1SUSAUX	192.168.113.???	192.168.115.1
     C1AUXEX	192.168.113.???	192.168.114.1
     C1AUXEY	192.168.113.???	192.168.11?.1
     C1PSL	192.168.113.???	192.168.11?.1
     C1VAC	192.168.113.???	192.168.11?.1
     C1???	192.168.113.???	192.168.11?.1

Remote Access

1. Set up SSH access

   • sudo apt -y install openssh-client

2. Optional: Set up remote desktop access

   • sudo apt -y install xrdp

3. Set up remote file system access
   • Run

     sudo apt-get install nfs-common
     mkdir /cvs/cds

   • Add line "chiara:/home/cds              /cvs/cds        nfs     rw,bg,nfsvers=3" to end of /etc/fstab and run

4. Install procserv for proper detaching of Modbus IOC systemd service

   sudo apt-get install procserv

Configure Acromag DAQ Modules

Initial configuration via USB is required before the units can be used on the network. This requires:

• USB-to-miniUSB cable
• Microsoft Windows (virtual) machine

• Configuration Utilities: 9500465D.zip

The above zip file contains the setup executable to install the configuration software for all different modules. Depending on the module, a different version of the utility needs to be launched after install. As during operation, the to-be-configured device needs to be powered by a 12-32V DC voltage source, either through the designated blue plug-in terminal on the bottom, or the clip-on dock on the rail.

On a virtual machine it may be necessary to open some ports to allow the guest OS to access the physical USB device.

In USB Settings:

add a corresponding filter:

The vendor ID and product ID can be found via lsusb

Network Configuration

The primary screen of the configuration utilities looks identical for all models:

Once the device is connected to the host via USB and the correct application has been launched, it will appear in the drop down menu. On the right the device IP and other network parameters can be set. Every host machine of the auxiliary DAQ acts as a node between the martian network and a local subnet to which the Acromag modules belong and which is not directly accessible from other machines on the martian network.

• For consistency, the ADC, DAC, and BIO modules occupy different IP address spaces:

  Module	Local IP Address Space
  ADC	20-39
  DAC	40-59
  BIO	60-79

• Hostname and IP on the local network are assigned according to the following scheme:

  Hostname	Local IP
  C1XXX-ADCYY	192.168.11X.YY+20
  C1XXX-DACYY	192.168.11X.YY+40
  C1XXX-BIOYY	192.168.11X.YY+60

• Other settings (of no practical significance) are
  • Gateway is the local IP of the node machine
  • DNS is that of Chiara on the martian network: 192.168.113.104

Configure XT unit Modbus driver

The EPICS binary modbusApp which launches the EPICS IOCs for the auxiliary DAQ system requires a .cmd instruction file which sets up the communication between the Acromag units and the EPICS IOC. The .cmd file is placed in /cvs/cds/caltech/target/HOSTNAME/HOSTNAME.cmd. It is largely similar to other EPICS binaries which launch an IOC, such as softIOC, with the addition that it can load the appropriate drivers for the modbus protocol prior to launching the IOC.

The file header consists of path definitions. At the current time there are no system variables pointing to these paths globally set, so the paths are explicitly defined.

epicsEnvSet("IOC", "c1XXX_iocconfig")   # insert proper hostname for XXX
epicsEnvSet("ARCH","linux-x86_64")
epicsEnvSet("TOP","/cvs/cds/rtapps/epics-3.14.12.2_long/modules/modbus")
epicsEnvSet("MDBTOP","/cvs/cds/rtapps/epics-3.14.12.2_long/modules/modbus")
dbLoadDatabase("$(MDBTOP)/dbd/modbus.dbd")
modbus_registerRecordDeviceDriver(pdbbase)

Each Acromag unit requires a set of three instructions which define the driver and Modbus parameters for the handshake during communication.

drvAsynIPPortConfigure(const char *portName,  # user-defined: used for subsequent referencing, 
                       const char *hostInfo,  # format: "IP-Address:Port". Standard port for Modbus is 502
                       unsigned int priority, # 
                       int noAutoConnect,     # 
                       int noProcessEos);     # 

modbusInterposeConfig(const char *portName,    # reference to portName created with drvAsynIPPortConfigure command
                      modbusLinkType linkType, # 
                      int timeoutMsec,         # define timeout for waiting for response from unit
                      int writeDelayMsec)      #

drvModbusAsynConfigure(portName,                # used by channel definitions in .db file to reference this unit)
                       tcpPortName,             # reference to portName created with drvAsynIPPortConfigure command
                       slaveAddress,            # 
                       modbusFunction,          # defines driver function for the unit (read register = 4, write register = 6, write single coil = 5) - see examples in next section
                       modbusStartAddress,      # ADC and binary channel numbering starts with 0, DAC channel numbering with 1
                       modbusLength,            # length in dataType units - see examples in next section
                       dataType,                # 4 = 16-bit signed integers (for A/D and D/A), 0 = binary (for BIO, duh)
                       pollMsec,                # how frequently to request a value in [ms]
                       plcType);                #

Examples

A generic example sequence would be

drvAsynIPPortConfigure("<TCP_PORT_NAME>","<UNIT_IP_ADDRESS>:502",0,0,1)
modbusInterposeConfig("<TCP_PORT_NAME>",0,5000,0)
drvModbusAsynConfigure("<PORT_NAME>","<TCP_PORT_NAME>",0,<modbusFunction>,<modbusStartAddress>,<modbusLength>,<dataType>,<pollMsec>,"Acromag")

Note that the default values for the last instruction differ between the different XT DAQ modules.

ADC Module XT1221:

drvAsynIPPortConfigure("c1susaux_adc00","192.168.115.20:502",0,0,1)
modbusInterposeConfig("c1susaux_adc00",0,5000,0)
drvModbusAsynConfigure("C1SUSAUX_ADC00","c1susaux_adc00",0,4,0,8,4,32,"Acromag")

DAC (+4x BIO) Module XT1541:

drvAsynIPPortConfigure("c1susaux_dac00","192.168.115.40:502",0,0,1)
modbusInterposeConfig("c1susaux_dac00",0,5000,0)
drvModbusAsynConfigure("C1SUSAUX_DAC00","c1susaux_dac00",0,6,1,8,4,1,"Acromag")
drvModbusAsynConfigure("C1SUSAUX_DAC00_BIO","c1susaux_dac00",0,5,0,4,0,1,"Acromag")   # Separate instruction for integrated BIO channels

BIO Modules XT1111 & XT1121:

drvAsynIPPortConfigure("c1susaux_bio00","192.168.115.60:502",0,0,1)
modbusInterposeConfig("c1susaux_bio00",0,5000,0)
drvModbusAsynConfigure("C1SUSAUX_BIO00","c1susaux_bio00",0,6,0,4,0,1,"Acromag")

After loading the drivers and setting up the modbus communication, the .cmd file instructs to load EPICS database files, just like other EPICS IOC starters. Like the .cmd file, the database files are located in /cvs/cds/caltech/target/HOSTNAME.

dbLoadDatabase("/cvs/cds/caltech/target/HOSTNAME/SUBSYSTEM.db")

Define EPICS channels

Note: Modbus/TCP is simply a protocol for sending commands via TCP that the XT units can interpret, as in read/write the correct register values to/from the channels. There may be multiple ways to define the channels have the same effect. The settings reported here have been found to work as intended.

The EPICS records in the .db files for the different DAQ channels are defined as follows (only fields relevant for proper addressing and conversion are shown):

XT1221 ADC Channels:

record(ai,"C1:<CHANNEL_NAME>")
{
        field(DTYP,"asynInt32")
        field(INP, "@asynMask(<PORT_NAME> <X> -16)MODBUS_DATA")   # <X>: 0-7 physical channel on Acromag unit
        field(LINR,"LINEAR")    # For count to float conversion: The ADC module converts +/-10V inputs to +/-30,000 counts (if legacy support is disabled, +/- 20,000 counts otherwise)
        field(EGUF,"10.923")    # Corresponds to +32767 in the above scale
        field(EGUL,"-10.923")   # Corresponds to -32767 in the above scale
}

XT1541 DAC Channels:

record(ao,"C1:<CHANNEL_NAME>")
{
        field(DTYP,"asynInt32")
        field(OUT, "@asynMask(<PORT_NAME>, <X>, -16)MODBUS_DATA")   # <X>: 0-7 physical channel on Acromag unit
        field(LINR,"LINEAR")    # For count to float conversion: The ADC module converts +/-10V inputs to +/-30,000 counts (if legacy support is disabled, +/- 20,000 counts otherwise)
        field(EGUF,"10.923")    # Corresponds to +32767 in the above scale
        field(EGUL,"-10.923")   # Corresponds to -32767 in the above scale
}

'''XT1541 BIO Channels:'''
record(bo, "<CHANNEL_NAME>")
{
        field(DTYP,"asynInt32")
        field(OUT,"@asynMask(<PORT_NAME>, <X>, -16)MODBUS_DATA")
}

XT1111 & XT1121 BIO Channels:

For binary input: Not yet confirmed

For binary output:

record(bo, "C1:<CHANNEL_NAME>")
{
        field(DTYP,"asynUInt32Digital")
        field(OUT,"@asynMask(<PORT_NAME>, <X>, 0x<Y>)")   # <X>: 0-3    <Y>: 1,2,4,or 8
}