Changes

v0.2.0

Use settings to setup connection parameters

The SecsHandler and GemHandler classes were initialized using the hsms configuration as parameters.

import secsgem

secsgem.GemHandler("10.211.55.33", 5000, False, 0, "test")

As new a new protocol and a new connection layer was added, this initialisation doesn’t fit all instances any more. For this reason new Settings classes were introduced:

import secsgem.common

import secsgem.hsms
import secsgem.secsi
import secsgem.secsitcp

hsms_settings = secsgem.hsms.HsmsSettings(
    device_type=secsgem.common.DeviceType.EQUIPMENT,
    connect_mode=secsgem.hsms.HsmsConnectMode.ACTIVE,
    address="127.0.0.1",
    port=5001,
)

secsi_settings = secsgem.secsi.SecsISettings(
    device_type=secsgem.common.DeviceType.HOST,
    port="COM1",
    speed=9600,
)

secsi_tcp_settings = secsgem.secsitcp.SecsITcpSettings(
    device_type=secsgem.common.DeviceType.EQUIPMENT,
    connect_mode=secsgem.secsitcp.SecsITcpConnectMode.CLIENT,
    address="127.0.0.1",
    port=5555,
)

# initialize handler with settings
import secsgem.gem

handler = secsgem.gem.GemEquipmentHandler(secsi_settings)

Add SECS-I and SECS-I over TCP support

The SECS-II and GEM handlers were built upon and inheriting the HSMS handler, making the communication and protocol mingled with the higher level message handling. This required separation of the communication and protocol from the message handling.

Communication layer

HsmsHandler <|-- SecsHandler : inherits SecsHandler <|-- GemHandler : inherits GemHandler <|-- GemEquipmentHandler : inherits GemHandler <|-- GemHostHandler : inherits

Old structure

The old class structure was quite linear, inheriting functionality from the basic class.

class Settings { +create_connection() +create_protocol() } SecsHandler <|-- GemHandler : inherits GemHandler <|-- GemEquipmentHandler : inherits GemHandler <|-- GemHostHandler : inherits Settings <- SecsHandler : initialized with Connection <-- Settings::create_connection : returns Protocol <-- Settings::create_protocol : returns Protocol <-- SecsHandler : uses Connection <- Protocol : uses

New Structure

Now the handlers are initialized with a Settings object, which in turn creates the Protocol and Communication classes used by the handler. This way the communication layer can be switched by simply passing different settings.

SecsHandler --> SecsISettings : create_protocol activate SecsISettings SecsISettings --> SecsIProtocol : constuct activate SecsIProtocol SecsIProtocol --> SecsISettings : create_connection activate SecsISettings SecsISettings --> SerialConnection : construct activate SerialConnection SerialConnection --> SecsISettings : created deactivate SerialConnection SecsISettings -> SecsIProtocol : return deactivate SecsISettings SecsIProtocol -> SecsISettings : created deactivate SecsIProtocol SecsISettings -> SecsHandler : return deactivate SecsISettings

Initialisation

Packets

The incoming data was encapsulated in a HsmsPacket object. But this was protocol specific.

Now every protocol type has an own message type, inherited from a base Message class. This base class defines the interface for the Messages. It uses a protocol specific header object inherited from Header and is made up of blocks using the Block class.

Renamed events

Due to the new communication layer (serial) and protocol (SECS-I) some events are renamed to match the use case:

old

new

hsms_connected

connected

hsms_selected

communicating

hsms_disconnected

disconnected

packet_received

message_received

Serialized message callback

Each message callback was called from a new thread to allow the receiver thread to be responsive for new packets. This could lead to messages not arriving in an expected order.

Receiver --> Handler : message received activate Handler Handler --> Thread : start thread activate Thread Handler -> Receiver : continue deactivate Handler Thread --> Callback : start callback activate Callback Callback -> Thread : callback finished deactivate Callback deactivate Thread

Old callback handling

Now incoming packets are added to a queue, which is processed by a single, separate thread. This allows the receiver thread to be responsive and the callbacks to be called in the order the packets are received.

participant Receiver participant Handler database Queue participant Processor participant Callback activate Processor Receiver --> Handler : message received activate Handler Handler --> Queue : queue message Handler -> Receiver : continue deactivate Handler Queue --> Processor : read queue Processor --> Callback : start callback activate Callback Callback -> Processor : callback finished deactivate Callback

New callback handling

Split big secsgem namespace and rename classes

All the classes were available in the toplevel secsgem namespace (e.g. secsgem.CallbackHandler). This global namespace made the classes easy to access, but it was pretty big and the classes had names that were redundant with the individual namespaces.

In this version the classes need to be imported and accessed using the sub-namespace.

Old:

import secgem
from secsgem import SecsHandler

secsgem.GemEquipmentHandler(...)
SecsHandler(...)

New:

import secgem.gem
from secsgem.secs import SecsHandler

secsgem.gem.GemEquipmentHandler(...)
SecsHandler(...)

Data item and stream/function classes are now auto-generated

As these classes are quite similar in code, the differences are extracted and included in yaml files in the data directory. (data/data_items.yaml and data/functions.yaml) The contents of the files in the directories secsgem/secs/data_items and secsgem/secs/functions are generated from these yaml files using templates. Also the class API documentation file is generated using the yaml files.

All generator data is located in the data directory in the project root, including the generator script (data/generate_data.py) and the template files (data/*.j2)

v0.3.0

Fail when initializing settings with invalid arguments

Due to the multi-layered settings structure the keyword arguments of the ‘Settings’ initializers were not checked for invalid arguments. A ValueError is now raised if an unknown parameter is passed to the settings constructor.

This is implemented for HsmsSettings, SecsISettings and SecsITcpSettings.

Also tests for the settings were added.

Add simple customization of equipment specific functions

The old, documented way for replacing / adding a customized function to a handler doesn’t work with 0.2.0 any more. So a new way to simply modify the function list for an equipment was added.

To allow equipment specific modifications of the function list, it was moved to the settings in 0.2.0. With this move a container for the function list was added, to allow simple lookups.

Now a function was added to this container, to update its function list. This will either add or update the function in the list. The container is accessible from the settings.

class UNITS_New(DataItemBase):
    name = "UNITS"

    __type__ = SecsVarDynamic  # changed
    __allowedtypes__ = [SecsVarU1, SecsVarU2, SecsVarU4, SecsVarI1, SecsVarI2, SecsVarI4, SecsVarString]  # changed

class SecsS01F12_New(secsgem.secs.SecsStreamFunction):
    _stream = 1
    _function = 12

    _data_format = [
        [
            SVID,
            SVNAME,
            UNITS_New
        ]
    ]

    _to_host = True
    _to_equipment = True  # Changed

    _has_reply = False
    _is_reply_required = False

    _is_multi_block = True

settings = secsgem.hsms.HsmsSettings()
settings.streams_functions.update(SecsS01F12_New)

handler = secsgem.gem.GemHostHandler(settings)

Add function definition language

Defining the data item for a function is a quite complex and confusing task. Wrapping the function definition in python data types is not very intuitive.

So a new description language was added, based on SML, which is known in combination with secs. For more information check the documentation (Secs Function Definition Language).

Attention

Custom data types are currently not supported with SFDL.

Old:

class SecsS06F08(SecsStreamFunction):
    _stream = 6
    _function = 8

    _data_format = [
        DATAID,
        CEID,
        [
            [
                "DS",
                DSID,
                [
                    [
                        "DV",
                        DVNAME,
                        DVVAL
                    ]
                ]
            ]
        ]
    ]

    _to_host = True
    _to_equipment = False

    _has_reply = False
    _is_reply_required = False

    _is_multi_block = True

New:

class SecsS06F08(SecsStreamFunction):
    _stream = 6
    _function = 8

    _data_format = """
        < L
            < DATAID >
            < CEID >
            < L DS
                < L
                    < DSID >
                    < L DV
                        < L
                            < DVNAME >
                            < DVVAL >
                        >
                    >
                >
            >
        >
    """

    _to_host = True
    _to_equipment = False

    _has_reply = False
    _is_reply_required = False

    _is_multi_block = True

Change data item and data item access

A new container for the data items was introduced. This container is usually accessible via the settings object and the handler, as it represents the equipment specific data items. This allows the customization of the data items on a per-equipment level.

Custom data item classes need new attributes to work.

class UNITS_New(DataItemBase):
    name = "UNITS"

    __type__ = SecsVarDynamic
    __allowedtypes__ = [SecsVarArray, SecsVarBoolean, SecsVarU1, SecsVarU2, SecsVarU4, SecsVarU8, SecsVarI1, SecsVarI2, SecsVarI4, SecsVarI8, \
        SecsVarF4, SecsVarF8, SecsVarString, SecsVarBinary]

The name attribute marks the name of the data item. This is required for lookup of the data item class using the settings / handler. It is also used when updating the data item using the container in the settings.

The _value attribute is the mapping used for the constants as a dictionary. It is used for lookup of the constants.

The downside of this implementation is that code suggestions in the IDE won’t work any more.

Recommendation: Start accessing the data items using the handler / settings.

The plan is to remove the function and data item classes and read the configuration directly from the yaml files. This means code written using the classes will need to be rechanged.

v0.4.0 [planned]

Rename session_id to device_id

Danger

This might break your code, changes to your code might be required.

Rename all session_id fields to device_id. This changes the naming of the variable to match the semi naming of the variable and to avoid confusion about this variable.

Update python minimum requirement to 3.9

As the version 3.8 is now EOL, the minimum requirement was raised to 3.9.

Add new classes for items

The old secs variable classes (secsgem.secs.variables.U4 etc.) are primarily used in the structuring for the function classes. As these function classes will be deprecated in favor of runtime parsed yaml files for function definition, these type of variables are not required any more.

The new Item classes mirror the old functionality for the data management without the need for structure definition. They also provide new functionalities for initializing them with data or getting data (eg sml).

The naming “Item” is also closer to the naming used by the SEMI SECS norm.

Load data item definition from yaml directly

Data item class files were generated using the yaml file. This made it necessary to run the class generator after changing the yaml file.

Instead of using the classes the data item definitions are now loaded from the yaml file directly. This makes the boilerplate class definition obsolete. No generator step is needed in between any more.

For now the data item classes are generated dynamically using the same base class as before. But the goal is to make the data item definition more flexible, without the use of data item classes.

The generated data item files are deleted from the package. This means that development IDEs won’t recognize the data items for autocompletion any more.

If you use data item classes directly, you need to update the way you get the data items. The data item list is available using the settings in the handler / protocol / connection.

Old:

lrack = secsgem.secs.data_items.LRACK.ACK

New:

lrack = secsgem.secs.data_items.DataItems().LRACK.ACK

# or

lrack = handler.settings.data_items.LRACK.ACK

The streams / functions now need the data_items as first parameter to initialize.

To find stream function initialization using the classes directly use the following regex:

SecsS\d\dF\d\d\s*\(

To find new setting / handler based function initialization use the following regex:

\.stream_function\(