diamondlightsource / pythonsoftioc

Embed an EPICS IOC in a Python process

pythonSoftIOC

This module allows an EPICS IOC with Python Device Support to be run from within the Python interpreter. Records can be programmatically created and arbitrary Python code run to update them and respond to caputs. It supports cothread and asyncio for concurrency. PVs are served over Channel Access and PVAccess.

PyPI	pip install softioc
Source code	https://github.com/DiamondLightSource/pythonSoftIOC
Documentation	https://DiamondLightSource.github.io/pythonSoftIOC
Changelog	https://github.com/DiamondLightSource/pythonSoftIOC/blob/master/CHANGELOG.rst

A simple example of the use of this library:

# Import the basic framework components.
from softioc import softioc, builder
import cothread

# Set the record prefix
builder.SetDeviceName("MY-DEVICE-PREFIX")

# Create some records
ai = builder.aIn('AI', initial_value=5)
ao = builder.aOut('AO', initial_value=12.45, on_update=lambda v: ai.set(v))

# Boilerplate get the IOC started
builder.LoadDatabase()
softioc.iocInit()

# Start processes required to be run after iocInit
def update():
    while True:
        ai.set(ai.get() + 1)
        cothread.Sleep(1)


cothread.Spawn(update)

# Finally leave the IOC running with an interactive shell.
softioc.interactive_ioc(globals())

See https://DiamondLightSource.github.io/pythonSoftIOC for more detailed documentation.

pythonsoftioc's Issues

Version reports 3.1 in 3.2

$ pipenv install softIoc==3.2 cothread
$ pipenv shell
$ pythonIoc --version
3.1
$ pipenv graph
cothread==2.17
  - numpy [required: Any, installed: 1.20.2]
softioc==3.2
  - epicscorelibs [required: >=7.0.6.99.1.0,<7.0.6.99.2, installed: 7.0.6.99.1.0]
    - numpy [required: Any, installed: 1.20.2]
    - setuptools [required: Any, installed: 52.0.0]
    - setuptools-dso [required: >=2.0a1, installed: 2.1]
  - epicsdbbuilder [required: >=1.4, installed: 1.4.3]
  - numpy [required: Any, installed: 1.20.2]

Fix Python2 main function

Before we drop Python2 support (see #61 ) we should do a release that fixes the Python2 main function.

Install Failed on Windows

I'm not sure what exactly happened, but I've successfully installed previous versions of softIoc. Python version is 3.10.4.

The entirety of the output follows, but I've snipped some highlights first.

It seemed there were two errors. One earlier having to do with a building a wheel:

"Building wheels for collected packages: softioc
  Building wheel for softioc (pyproject.toml) ... error
  error: subprocess-exited-with-error
  × Building wheel for softioc (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [103 lines of output]"

then later on:

  "packages\setuptools_dso\dsocmd.py", line 88, in expand_sources
          raise RuntimeError("Missing source file: %s"%src)
      RuntimeError: Missing source file: iocStats\devIocStats\os\default\osdCpuUsage.c
      [end of output]
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for softioc
Failed to build softioc
ERROR: Could not build wheels for softioc, which is required to install pyproject.toml-based projects"

I suspect the latter has to do with the former.

Regards,
Rob (CLS)

Full output:

pip install softioc
Collecting softioc
  Using cached softioc-4.0.1.tar.gz (58 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting numpy
  Using cached numpy-1.22.4-cp310-cp310-win_amd64.whl (14.7 MB)
Collecting epicscorelibs<7.0.6.99.3,>=7.0.6.99.2.0
  Using cached epicscorelibs-7.0.6.99.2.0-cp310-cp310-win_amd64.whl (2.6 MB)
Collecting epicsdbbuilder>=1.4
  Using cached epicsdbbuilder-1.5-py3-none-any.whl (23 kB)
Requirement already satisfied: setuptools in c:\users\patelk\appdata\local\programs\python\python310\lib\site-packages (from epicscorelibs<7.0.6.99.3,>=7.0.6.99.2.0->softioc) (58.1.0)
Collecting setuptools-dso>=2.0a1
  Using cached setuptools_dso-2.5-py2.py3-none-any.whl (20 kB)
Building wheels for collected packages: softioc
  Building wheel for softioc (pyproject.toml) ... error
  error: subprocess-exited-with-error
  × Building wheel for softioc (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [103 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-310
      creating build\lib.win-amd64-cpython-310\softioc
      copying softioc\alarm.py -> build\lib.win-amd64-cpython-310\softioc
      copying softioc\asyncio_dispatcher.py -> build\lib.win-amd64-cpython-310\softioc
      copying softioc\builder.py -> build\lib.win-amd64-cpython-310\softioc
      copying softioc\device.py -> build\lib.win-amd64-cpython-310\softioc
      copying softioc\device_core.py -> build\lib.win-amd64-cpython-310\softioc
      copying softioc\fields.py -> build\lib.win-amd64-cpython-310\softioc
      copying softioc\imports.py -> build\lib.win-amd64-cpython-310\softioc
      copying softioc\pvlog.py -> build\lib.win-amd64-cpython-310\softioc
      copying softioc\pythonSoftIoc.py -> build\lib.win-amd64-cpython-310\softioc
      copying softioc\softioc.py -> build\lib.win-amd64-cpython-310\softioc
      copying softioc\_version_git.py -> build\lib.win-amd64-cpython-310\softioc
      copying softioc\__init__.py -> build\lib.win-amd64-cpython-310\softioc
      copying softioc\__main__.py -> build\lib.win-amd64-cpython-310\softioc
      copying softioc\access.acf -> build\lib.win-amd64-cpython-310\softioc
      copying softioc\device.dbd -> build\lib.win-amd64-cpython-310\softioc
      copying softioc\devIocStats.dbd -> build\lib.win-amd64-cpython-310\softioc
      creating build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\access.db -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\access.doc -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\ioc.template -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocAdminRTEMS.substitutions -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocAdminScanMon.substitutions -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocAdminSoft.substitutions -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocAdminVxWorks.substitutions -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocCluster.template -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocEnvVar.template -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocGeneralTime.template -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocQueue.db -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocRTEMSOnly.template -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocRTOS.template -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocScanMon.template -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocScanMonSum.template -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocVxWorksOnly.template -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      copying softioc\iocStatsDb\Makefile -> build\lib.win-amd64-cpython-310\softioc\iocStatsDb
      running build_ext
      Traceback (most recent call last):
        File "C:\Users\patelk\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 363, in <module>
          main()
        File "C:\Users\patelk\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "C:\Users\patelk\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 261, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\build_meta.py", line 244, in build_wheel
          return self._build_with_temp_dir(['bdist_wheel'], '.whl',
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\build_meta.py", line 229, in _build_with_temp_dir
          self.run_setup()
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\build_meta.py", line 281, in run_setup
          super(_BuildMetaLegacyBackend,
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\build_meta.py", line 174, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 103, in <module>
          setup(
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools_dso\__init__.py", line 60, in setup
          _setup(**kws)
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\__init__.py", line 87, in setup
          return distutils.core.setup(**attrs)
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 148, in setup
          return run_commands(dist)
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 163, in run_commands
          dist.run_commands()
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 967, in run_commands
          self.run_command(cmd)
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\dist.py", line 1229, in run_command
          super().run_command(command)
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 986, in run_command
          cmd_obj.run()
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\wheel\bdist_wheel.py", line 299, in run
          self.run_command('build')
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\dist.py", line 1229, in run_command
          super().run_command(command)
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 986, in run_command
          cmd_obj.run()
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\_distutils\command\build.py", line 136, in run
          self.run_command(cmd_name)
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\dist.py", line 1229, in run_command
          super().run_command(command)
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 986, in run_command
          cmd_obj.run()
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools_dso\dsocmd.py", line 509, in run
          _build_ext.run(self)
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\command\build_ext.py", line 79, in run
          _build_ext.run(self)
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 339, in run
          self.build_extensions()
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 448, in build_extensions
          self._build_extensions_serial()
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 473, in _build_extensions_serial
          self.build_extension(ext)
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools_dso\dsocmd.py", line 512, in build_extension
          expand_sources(self, ext.sources)
        File "C:\Users\patelk\AppData\Local\Temp\pip-build-env-dm1jp3ze\overlay\Lib\site-packages\setuptools_dso\dsocmd.py", line 88, in expand_sources
          raise RuntimeError("Missing source file: %s"%src)
      RuntimeError: Missing source file: iocStats\devIocStats\os\default\osdCpuUsage.c
      [end of output]
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for softioc
Failed to build softioc
ERROR: Could not build wheels for softioc, which is required to install pyproject.toml-based projects

Unable to set StringOut record values

I seem to be unable to set the value of a StringOut record programmatically by calling .set(). This program demonstrates the issue:

# Import the basic framework components.
from softioc import softioc, builder, asyncio_dispatcher

# Create an asyncio dispatcher, the event loop is now running
dispatcher = asyncio_dispatcher.AsyncioDispatcher()

# Set the record prefix
builder.SetDeviceName("MY-DEVICE-PREFIX")

# Create some records
strout = builder.stringOut("FLIBBLE", initial_value="ABC")

# Boilerplate get the IOC started
builder.LoadDatabase()
softioc.iocInit(dispatcher)

strout.set("DEF")

# Finally leave the IOC running with an interactive shell.
softioc.interactive_ioc(globals())

The strout.set() line generates this exception:

 File "/home/eyh46967/dev/pythonSoftIoc/docs/examples/trial_stringout.py", line 17, in <module>
    strout.set("DEF")
  File "/home/eyh46967/dev/pythonSoftIoc/softioc/device.py", line 212, in set
    datatype, length, data, array = self.value_to_dbr(value)
  File "/home/eyh46967/dev/pythonSoftIoc/softioc/device.py", line 186, in value_to_dbr
    dbrtype = self.NumpyCharCodeToDbr[value.dtype.char]
KeyError: 'U'

The device.py code appears to be trying to convert the string value into a numpy array on line 173:

value = numpy.require(value, requirements = 'C')

Examining this array gives a value.dtype.char of U, which is unknown to the NumpyCharCodeToDbr mapping dictionary.

Can't figure out how to change IOC prompt.

I'm trying to customize iocApp prompts similar to:

epicsEnvSet IOCSH_PS1 "AARMS (s71500)> "

How do I accomplish that with pythonSoftIOC....?

Cheers,
Rob

Add caput callback support

This requires some research

boolOut records require ZNAM and ONAM if initial_value is set

If you create a boolOut record, specify an initial_value, and do NOT specify ZNAM and ONAM explicitly the resultant record has no value. I'm not sure if this is just unclear documentation or an actual bug.

# Import the basic framework components.
from softioc import softioc, builder, asyncio_dispatcher

# Create an asyncio dispatcher, the event loop is now running
dispatcher = asyncio_dispatcher.AsyncioDispatcher()

# Set the record prefix
builder.SetDeviceName("MY-DEVICE-PREFIX")

# Create some records
bo1 = builder.boolOut('BO1', initial_value=1)
bo2 = builder.boolOut('BO2', ZNAM=0, ONAM=1, initial_value=1)

# Boilerplate get the IOC started
builder.LoadDatabase()
softioc.iocInit(dispatcher)

# Finally leave the IOC running with an interactive shell.
softioc.interactive_ioc(globals())

And on the command line we see BO1 has no value, while BO2 does:

[...]$ caget MY-DEVICE-PREFIX:BO1
MY-DEVICE-PREFIX:BO1           
[...]$ caget MY-DEVICE-PREFIX:BO2
MY-DEVICE-PREFIX:BO2           1

Weird channel access problem within Python softioc process

From thread tech talk.

A simplified example of the problem:

1. Import the pyepics and softioc libraries.
2. Start a softioc with a channel (call it ‘prefix:long’ in my example).
3. Start a thread that periodically updates the channel with a new value.
4. Start another thread that tries to access this channel, either with epics.PV() or epics.caget().
5. Channel access in thread 2 always times out, forever.

Today I found a “solution” to this that does not make sense: if I start the softioc (the call to softioc.iotInit()) in the first thread, and not in the main process, everything works as expected.

Attached example code:

import os
import time
import threading

os.environ['EPICS_CA_SERVER_PORT'] = '5100'
os.environ['EPICS_CA_ADDR_LIST'] = 'localhost:5100'
os.environ['EPICS_CA_AUTO_ADDR_LIST'] = 'NO'
 
try:
    import epicscorelibs.path.pyepics
except ImportError:
    pass

import epics
from softioc import softioc, builder

# Create a long record for this demo
long = builder.longOut('prefix:long', initial_value=1)
builder.LoadDatabase()
softioc.iocInit()

def thread1():
    while True:
        val = long.get()
        long.set(val + 1)
        print(f'thread1: {val}')
        time.sleep(1)

def thread2():
    pv = epics.PV('prefix:long')
    while True:
        val = pv.get()
        print(f'thread2: {val}')
        time.sleep(1)

t1 = threading.Thread(target=thread1, daemon=True)
t2 = threading.Thread(target=thread2, daemon=True)

t1.start()
t2.start()

time.sleep(10)

If iocInit is moved into thread1() the example works.

What is a "Publishable IOC"?

Hope this is the right place for this question.....

Saw a reference to "Publishable IOC" in the docs. Is that a Diamond-specific definition or EPICS v4 or something else? What is it?

Cheers,
Tanner

Upgrade to Anaconda Python 3.7 breaks

On CentOS 7 systems using Anaconda Python 3.6 in /usr/local/anaconda, we have been successfully using the pythonIoc for some time now. The Anaconda package is installed via its own installer, not by any yum/rpm installers.

Recently I built another system (CentOS 7.7) with the latest anaconda3-2019.10 installed in the same place, /usr/local/anaconda. I created a new, clean build of pythonIoc. However, invocation of pythonIoc fails to find libraries and core dumps:

$ ./pythonIoc
Fatal Python error: initfsencoding: Unable to get the locale encoding
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007f1a8468a740 (most recent call first):
Abort (core dumped)

We investigated this error and learned that the errors relate to how Python searching for the standard libraries. Troubleshooting articles suggest setting PYTHONHOME as defined here: https://docs.python.org/3.7/using/cmdline.html#environment-variables

setenv PYTHONHOME /usr/local/anaconda

This appears to solve the problem.

$ ./pythonIoc
Python 3.7.4 (default, Aug 13 2019, 20:39:44)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

It's not clear why this was not a problem in the past. PYTHONHOME has been around since the Python 2 days and I've never had to set it before.

We suggest maybe using the CONFIG_SITE file to establish the base Python directory, then create PYTHONHOME in pythonIoc.in based on that.

Provide mechanism to get record type

It would be convenient to have some method on a RecordWrapper that allows retrieving the record's type. For example, a get_type() record might return a string ao, to say that this record is an Analog Out type, or bi for Boolean In, etc.

This would allow users to query RecordWrapper instances dynamically to know what keyword arguments set() allows, for example.

[question] will this package be published to PyPi?

The README and docs both require to manually install this package. I wonder if there is a plan to publish it to PyPi so that we can use the pip tool to manage this package.

If yes, please tell the schedule. If no, please tell the reasoning behind.

// This tool is awesome! Thx for your work!

Improving code coverage

Looking at, for example Codecov:device.py, there is some low hanging fruit that could easily be added to the tests.

• Setting the alarm with .set_alarm() and reading back the currently set value on an In record with .get(); both these functions are simple to call and check for side effects.
• I'm a little surprised to see that we have no Out records without on_update set (this would cover L106)
• Are we actually triggering Out record processing, or has the code coverage engine failed to spot that lines 130 to 146 are being executed?
• Are we testing calling .set() on Out records at all? This would trigger a lot of code that's uncovered at present.

Command line interface script should be optional

If no script present, just run the executable to get an interactive python environment

Inherited methods/properties from EpicsDBBuilder are not documented

There's currently no documentation that says what is (and is not) inherited from EpicsDBBuilder and so what is valid to call on a RecordWrapper instance. This should be improved.

Unable to run waveform.set() before softioc.iocInit()

In a patch from 2019 (CGP-182, 944e072), a change was made to the waveform 'set' method. As part of this change, it now uses the 'dtype' attribute.

This attribute is only set when the 'init_record' method is called, which is triggered by the softioc.iocInit() call for IOC startup.

Here is my particular use-case:

The module I am working on has its behaviour split into a number of separate 'servers', each with their own EPICS database setup and associated logic. The 'softioc.iocInit()' method is only called by the top-level program, after each of these servers has had a chance to run its own database setup.

The unit tests do not call 'softioc.iocInit()'. They can, for example, allow the servers to run waveform.set() and then check the parameters afterwards.

This has problems after the above bug-fix, as dtype is not set. It seems sensible to me that unit tests should not need to have a running IOC in order to operate. What is your opinion on allowing get() and set() functions to be called before iocInit()? Do you think the above is a good model for unit testing? If it's not, what alternatives should be used?

Add vscode config

Be able to force Out records into an error state

Currently there is no explicit way to force an Out record into an error state. The set() method on In records provides the severity and alarm keyword arguments (as well as the set_alarm() method), but there is no equivalent mechanism for Out records. Providing this would be useful for the PandABlocks-client PythonSoftIOC integration, as PandA can report fields as being in_error, and the associated records are often Out types.

Versioning with _version_git breaks if softioc is installed

If a module installs softioc, and also uses VersionGit for versioning, the version number appears to be set to that of softioc when making a test release. This may also affect released modules.

Steps to reproduce (using Diamond tools):

dls-start-new-module.py -a python3 -n dls_example_module
cd dls_example_module/
pipenv install -d
pipenv install softioc
dls-py3 install-into-prefix
ls prefix/lib/python3.7/site-packages/

This will result in a module which shows the version to be 3.1, which is the latest release of softioc:

dls_example_module  dls_example_module-3.1.dist-info

What I have found so far:

• It looks like _version_git in the your new module doesn't get called when dls-py3 install-into-prefix is called.
• On a clean repository, pipenv run python -m setup bdist_wheel will get the correct version (last tag + commit hash etc). If you run dls-py3 install-into-prefix first, the wheel will get stuck with the softioc version
• When running _version_git after install, it appears that GIT_REFS and GIT_SHA1 are set to the values from softioc
• This exists on (at least) versions 3.0, 3.0b2 and 3.1

Some Clarifications Regarding aIn records

It seems there's a variety of ways to define ioc records and I'm looking to confirm or clarify my understanding thus far.

Two main categories are ones using the special PythonIOC device and the lower level ones in builder.records.

I defined:

ai2 = builder.records.ai('AI2', VAL="5", INP="float1 CP MS", SCAN="Passive", PINI="YES")

and updates were seen with a camonitor running in another console.

Conversely:

ai1 = builder.aIn('AI', initial_value=5, INP="float1 CP MS", SCAN="Passive", PINI="YES")

did not update (verified using both caget and camonitor from another console)

So, I set up a monitor on float1 (as per the tutorials) and wrote the value from the callback function. Updates still did not go out. I then changed the SCAN to "I/O Intr" for ai2 and updates started going out.

Do all these "higher-level" records require SCAN="I/O Intr"?
Do they also require that the value is manually set in Python (i.e. ai.set(value))......?

Or should I be updated ai1 via an ao1's on_update handler....? Is there a "best practice" for what I'm attempting?

Thanks so much for this intriguing tool/utility!!! It holds a lot of promise for us here. :)

PINI flag is not set on In records when .set() is used

In builder.py's _in_record function we see this code:

if 'initial_value' in fields:
        fields.setdefault('PINI', 'YES')

There is no equivalent code for when .set() is called. This means that In records do not process on startup correctly such that caget sees the value. For example run the following IOC in a terminal:

# Import the basic framework components.
from softioc import softioc, builder, asyncio_dispatcher
import numpy
# Create an asyncio dispatcher, the event loop is now running
dispatcher = asyncio_dispatcher.AsyncioDispatcher()

# Set the record prefix
builder.SetDeviceName("PREFIX")

# Create some records
li = builder.longIn("TEST")

li.set(10)

# Boilerplate get the IOC started
builder.LoadDatabase()
softioc.iocInit(dispatcher)


# Finally leave the IOC running with an interactive shell.
softioc.interactive_ioc(globals())

And from another do caget:

[...]$ caget PREFIX:TEST
PREFIX:TEST                    0

And we see that the .set() has not taken effect.

Waveforms with empty array initial values do not update as expected

When using a WaveformOut record and doing .set() and .get(), the behaviour is not as expected when the initial_value is an empty numpy array:

# Import the basic framework components.
from softioc import softioc, builder, asyncio_dispatcher
import numpy
# Create an asyncio dispatcher, the event loop is now running
dispatcher = asyncio_dispatcher.AsyncioDispatcher()

# Set the record prefix
builder.SetDeviceName("PREFIX")

# Create some records
wo = builder.WaveformOut("TEST", initial_value=numpy.array([]))

# Boilerplate get the IOC started
builder.LoadDatabase()
softioc.iocInit(dispatcher)

print(wo.get())
wo.set(numpy.array([10]))
print(wo.get())

# Finally leave the IOC running with an interactive shell.
softioc.interactive_ioc(globals())

This program prints:

...
iocRun: All initialization complete
[]
[]

If you change the initial_value to numpy.array([1]) then the program prints out the expected results:

...
iocRun: All initialization complete
[1]
[10]

Apologies if this is a duplicate of an existing issue, but it certainly is a novel symptom.

asyncio vs cothreads

It seems to me that asyncio will work on both Windows and *nix.

Is there a compelling reason to use cothreads in Unix-land....?

Remove Python2 support

While PythonSoftIOC can support Python2, the code in __main__.py does not work in Python2.

As per discussion with @Araneidae and @thomascobb it's probably not worth fixing it, instead this issue will be to remove Python2 support entirely.

This will involve both a lot of code cleanup (there's a lot of Python version checks littered throughout the code) as well as reviewing any documentation about version support.

String record handling data truncation

When creating a string record, if you set an initial_value that is longer than 39 characters the data is silently truncated. It would be useful if there was at least a warning/error of some kind emitted when this occurs.

In this example note that the trailing "!" of the string is missing:

# Import the basic framework components.
from softioc import softioc, builder, asyncio_dispatcher

# Create an asyncio dispatcher, the event loop is now running
dispatcher = asyncio_dispatcher.AsyncioDispatcher()

# Set the record prefix
builder.SetDeviceName("PREFIX")

# Create some records
bo1 = builder.stringOut('SO', initial_value="hello this is long sentence of 39 chars!")

# Boilerplate get the IOC started
builder.LoadDatabase()
softioc.iocInit(dispatcher)

# Finally leave the IOC running with an interactive shell.
softioc.interactive_ioc(globals())

Getting the value:

[...]$ caget PREFIX:SO
PREFIX:SO                      hello this is long sentence of 39 chars

Trying to build debian package from source

Hey,

I am trying to build the softioc from source to create a debian package.

I am able to build the dependencies from source:

• setuptools-dso
• epicscorelibs
• epicsdbbuilder

But during the build process for softioc I run into the following exception:

RuntimeError: Unable to find DSO epicscorelibs.lib.qsrv needed by extension softioc._extension

More detailed log:

dpkg-buildpackage: info: source package softioc
dpkg-buildpackage: info: source version 3.2-1
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Michael Abbott <[email protected]>
dpkg-buildpackage: info: host architecture amd64
dpkg-source: info: using options from softioc-3.2/debian/source/options: --extend-diff-ignore=\.egg-info$
dh build --with python3 --buildsystem=python_distutils
   dh_update_autotools_config -O--buildsystem=python_distutils
   dh_auto_configure -O--buildsystem=python_distutils
   debian/rules override_dh_auto_build
make[1]: Entering directory '/builds/python-tools/deb-builder/deb_dist/softioc-3.2'
python3 setup.py build --force
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/softioc
copying softioc/__main__.py -> build/lib.linux-x86_64-3.8/softioc
copying softioc/fields.py -> build/lib.linux-x86_64-3.8/softioc
copying softioc/builder.py -> build/lib.linux-x86_64-3.8/softioc
copying softioc/alarm.py -> build/lib.linux-x86_64-3.8/softioc
copying softioc/softioc.py -> build/lib.linux-x86_64-3.8/softioc
copying softioc/device.py -> build/lib.linux-x86_64-3.8/softioc
copying softioc/pythonSoftIoc.py -> build/lib.linux-x86_64-3.8/softioc
copying softioc/asyncio_dispatcher.py -> build/lib.linux-x86_64-3.8/softioc
copying softioc/_version_git.py -> build/lib.linux-x86_64-3.8/softioc
copying softioc/imports.py -> build/lib.linux-x86_64-3.8/softioc
copying softioc/__init__.py -> build/lib.linux-x86_64-3.8/softioc
copying softioc/device_core.py -> build/lib.linux-x86_64-3.8/softioc
copying softioc/pvlog.py -> build/lib.linux-x86_64-3.8/softioc
copying softioc/access.acf -> build/lib.linux-x86_64-3.8/softioc
copying softioc/device.dbd -> build/lib.linux-x86_64-3.8/softioc
copying softioc/devIocStats.dbd -> build/lib.linux-x86_64-3.8/softioc
creating build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/iocAdminVxWorks.substitutions -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/iocScanMonSum.template -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/iocAdminScanMon.substitutions -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/iocCluster.template -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/iocScanMon.template -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/iocRTEMSOnly.template -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/iocQueue.db -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/iocGeneralTime.template -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/Makefile -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/iocVxWorksOnly.template -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/iocAdminSoft.substitutions -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/iocAdminRTEMS.substitutions -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/iocRTOS.template -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/access.doc -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/access.db -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/iocEnvVar.template -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
copying softioc/iocStatsDb/ioc.template -> build/lib.linux-x86_64-3.8/softioc/iocStatsDb
running build_ext
make[1]: Leaving directory '/builds/python-tools/deb-builder/deb_dist/softioc-3.2'


Standard error:
dpkg-buildpackage: warning: using a gain-root-command while being root
 dpkg-source --before-build .
 debian/rules build
dh_auto_configure: warning: Please use the third-party "pybuild" build system instead of python-distutils
dh_auto_configure: warning: This feature will be removed in compat 12.
Traceback (most recent call last):
  File "setup.py", line 103, in <module>
    setup(
  File "/usr/lib/python3/dist-packages/setuptools_dso/__init__.py", line 58, in setup
    _setup(**kws)
  File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 144, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.8/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib/python3/dist-packages/setuptools_dso/dsocmd.py", line 511, in run
    _build_ext.run(self)
  File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 87, in run
    _build_ext.run(self)
  File "/usr/lib/python3.8/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/usr/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/usr/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/usr/lib/python3/dist-packages/setuptools_dso/dsocmd.py", line 522, in build_extension
    self.dso2lib_pre(ext)
  File "/usr/lib/python3/dist-packages/setuptools_dso/dsocmd.py", line 169, in dso2lib_pre
    raise RuntimeError("Unable to find DSO %s needed by extension %s"%(dso, ext.name))
RuntimeError: Unable to find DSO epicscorelibs.lib.qsrv needed by extension softioc._extension
make[1]: *** [debian/rules:14: override_dh_auto_build] Error 1
make: *** [debian/rules:7: build] Error 2
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2

I also tried to install epics base first, but it did not help. Any advice?

Inconsistencies around defaults for Analog and Long records

The fields which are defaulted for analogIn/Out and longIn/Out records in builder.py are inconsistent in various ways:

• aOut will default the DRVL and DEVH fields, but longOut will not
• long records specify EGU but analog records do not

The documentation also reflects these same inconsistencies and should be adjusted to whatever fix is made.

python path for user library seems to be broken

/usr/bin/ld: /usr/lib/libpython3.6m.a(faulthandler.o): undefined reference to symbol 'pthread_sigmask@@GLIBC_2.2.5'
/lib/x86_64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
/home/test/myepics/base/configure/RULES_BUILD:230: recipe for target 'softIoc' failed
make[2]: *** [softIoc] Error 1
make[2]: Leaving directory '/home/test/Desktop/pythonIoc/softIocApp/O.linux-x86_64'
/home/test/myepics/base/configure/RULES_ARCHS:58: recipe for target 'install.linux-x86_64' failed
make[1]: *** [install.linux-x86_64] Error 2
make[1]: Leaving directory '/home/test/Desktop/pythonIoc/softIocApp'
/home/test/myepics/base/configure/RULES_DIRS:85: recipe for target 'softIocApp.install' failed
make: *** [softIocApp.install] Error 2

After following directions, I get to the point where the user makes the BUILD_DOCS=, and it spits out this error.
It cannot find the directory for python. I have python installed and checked for any version problems.

Provide an add_alias() method to ease alias creation

Currently a record, inside a RecordWrapper, can have aliases added to it by calling add_alias(). However this is a bit awkward as you have to provide not just the alias'd name but also the device prefix. This often means an application wanting to create aliases has to keep the record prefix accessible everywhere. This is obviously somewhat awkward and could be streamlined by having this logic handled for us in PythonSoftIOC.

Setting Strings as values for WaveformOut has multiple issues

Attempting to create a WaveformOut record with a String value has multiple issues regarding how PythonSoftIOC attempts to convert it into a numpy array. The failure modes are different depending on how you try and set the value.

Using initial_value="Some string" and FTVL="CHAR" you get this exception:

len() of unsized object
  File "/scratch/eyh46967/pipenv/PandABlocks-client-bVxlyL_p/lib/python3.7/site-packages/softioc/builder.py", line 167, in _waveform
    length = len(value)
  File "/scratch/eyh46967/pipenv/PandABlocks-client-bVxlyL_p/lib/python3.7/site-packages/softioc/builder.py", line 192, in WaveformOut
    _waveform(value, fields)
  File "/home/eyh46967/dev/PandABlocks-client/pandablocks/ioc.py", line 618, in _waveform_rec
    **kwargs,
  File "/home/eyh46967/dev/PandABlocks-client/pandablocks/ioc.py", line 580, in __init__
    DESC="Format string used for file naming",
  File "/home/eyh46967/dev/PandABlocks-client/pandablocks/ioc.py", line 1723, in create_block_records
    _HDF5RecordController(self._client)

Creating the record and immediately doing .set("some string") on the record seems to be ignored entirely with no exception but also no change to the record's value.

Creating the record and immediately doing .set("%abc") gives the following exception:

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 232, in 'calling callback function'
  File "/scratch/eyh46967/pipenv/PandABlocks-client-bVxlyL_p/lib/python3.7/site-packages/softioc/device_core.py", line 191, in _init_record
    return self.init_record(record)
  File "/scratch/eyh46967/pipenv/PandABlocks-client-bVxlyL_p/lib/python3.7/site-packages/softioc/device.py", line 310, in init_record
    return self.__super.init_record(record)
  File "/scratch/eyh46967/pipenv/PandABlocks-client-bVxlyL_p/lib/python3.7/site-packages/softioc/device.py", line 125, in init_record
    self._write_value(record, self._value)
  File "/scratch/eyh46967/pipenv/PandABlocks-client-bVxlyL_p/lib/python3.7/site-packages/softioc/device.py", line 321, in _write_value
    value = numpy.require(value, dtype=self.dtype)
  File "/scratch/eyh46967/pipenv/PandABlocks-client-bVxlyL_p/lib/python3.7/site-packages/numpy/core/_asarray.py", line 380, in require
    return asanyarray(a, dtype=dtype)
  File "/scratch/eyh46967/pipenv/PandABlocks-client-bVxlyL_p/lib/python3.7/site-packages/numpy/core/_asarray.py", line 171, in asanyarray
    return array(a, dtype, copy=False, order=order, subok=True)
ValueError: invalid literal for int() with base 10: '%abc'

From observation it also appears that doing a delayed .set() after IOC initialisation will result in similar errors, but again in a different place.

It should be noted that WaveformIn records have some attempts at handling this issue - device.py L353 has value = numpy.fromstring(value + "\0", dtype="uint8") which will work, but emits this warning:
_main__:1: DeprecationWarning: The binary mode of fromstring is deprecated, as it behaves surprisingly on unicode inputs. Use frombuffer instead

The workaround is to create the record and do .set(np.frombuffer("%s/%s_%d.h5".encode() + b"\0", dtype=np.uint8)) or similar.

Note that I cannot provide the value during record creation using either value or initial_value, as I need to set a maximum length of the record which is bigger than the length of the initial value. This is covered in issue #37

2D Data

Hi,
i'm quite new to EPICS, so this is maybe a dumb question:

How can i use a 2D array as record in pythonSoftIOC?
This would be much simpler than the EPICS areaDetector in C as i already have my video frames in python.

Thanks!

Allow users to reset the list of created records

epicsdbbuilder exports the method ResetRecords(), documentation here, which allows the known list of records to be forgotten.

PythonSoftIOC maintains its own dictionary of records in RecordLookup, which appears to offer the same protections are epicsdbbuilder but offers no reset mechanism.

The situation I'm in is that I'm trying to test my record creation (note: not actually starting the IOC, just testing the creation). An epicsdbbuilder error about duplicat records is encountered first, but calling ResetRecords() allows me to work around that, within the published API. However I have no (sensible) way to reset the dictionary that PythonSoftIOC maintains. This would force me to create a multi-process test framework, which in quick tests is 2x-3x slower than running in a single process.

Sdist not packaged properly for windows

At the moment we make the sdist on Linux, which pulls in sources for Linux only:
https://github.com/dls-controls/pythonSoftIOC/blob/master/setup.py#L46

DevIocStats requires different sources for windows, so we might be able to get away with adding them as package_data?

C:\Users\guent>pip install softioc
Collecting softioc
  Using cached softioc-4.0.1.tar.gz (58 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting epicsdbbuilder>=1.4
  Using cached epicsdbbuilder-1.5-py3-none-any.whl (23 kB)
Requirement already satisfied: numpy in c:\users\guent\appdata\local\programs\python\python310\lib\site-packages (from softioc) (1.22.2)
Requirement already satisfied: epicscorelibs<7.0.6.99.3,>=7.0.6.99.2.0 in c:\users\guent\appdata\local\programs\python\python310\lib\site-packages (from softioc) (7.0.6.99.2.0)
Requirement already satisfied: setuptools in c:\users\guent\appdata\local\programs\python\python310\lib\site-packages (from epicscorelibs<7.0.6.99.3,>=7.0.6.99.2.0->softioc) (58.1.0)
Requirement already satisfied: setuptools-dso>=2.0a1 in c:\users\guent\appdata\local\programs\python\python310\lib\site-packages (from epicscorelibs<7.0.6.99.3,>=7.0.6.99.2.0->softioc) (2.5)
Building wheels for collected packages: softioc
  Building wheel for softioc (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for softioc (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [97 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-3.10
      creating build\lib.win-amd64-3.10\softioc
      copying softioc\alarm.py -> build\lib.win-amd64-3.10\softioc
      copying softioc\asyncio_dispatcher.py -> build\lib.win-amd64-3.10\softioc
      copying softioc\builder.py -> build\lib.win-amd64-3.10\softioc
      copying softioc\device.py -> build\lib.win-amd64-3.10\softioc
      copying softioc\device_core.py -> build\lib.win-amd64-3.10\softioc
      copying softioc\fields.py -> build\lib.win-amd64-3.10\softioc
      copying softioc\imports.py -> build\lib.win-amd64-3.10\softioc
      copying softioc\pvlog.py -> build\lib.win-amd64-3.10\softioc
      copying softioc\pythonSoftIoc.py -> build\lib.win-amd64-3.10\softioc
      copying softioc\softioc.py -> build\lib.win-amd64-3.10\softioc
      copying softioc\_version_git.py -> build\lib.win-amd64-3.10\softioc
      copying softioc\__init__.py -> build\lib.win-amd64-3.10\softioc
      copying softioc\__main__.py -> build\lib.win-amd64-3.10\softioc
      copying softioc\access.acf -> build\lib.win-amd64-3.10\softioc
      copying softioc\device.dbd -> build\lib.win-amd64-3.10\softioc
      copying softioc\devIocStats.dbd -> build\lib.win-amd64-3.10\softioc
      creating build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\access.db -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\access.doc -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\ioc.template -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocAdminRTEMS.substitutions -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocAdminScanMon.substitutions -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocAdminSoft.substitutions -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocAdminVxWorks.substitutions -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocCluster.template -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocEnvVar.template -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocGeneralTime.template -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocQueue.db -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocRTEMSOnly.template -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocRTOS.template -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocScanMon.template -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocScanMonSum.template -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\iocVxWorksOnly.template -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      copying softioc\iocStatsDb\Makefile -> build\lib.win-amd64-3.10\softioc\iocStatsDb
      running build_ext
      Traceback (most recent call last):
        File "C:\Users\guent\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 363, in <module>
          main()
        File "C:\Users\guent\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "C:\Users\guent\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 261, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\build_meta.py", line 244, in build_wheel
          return self._build_with_temp_dir(['bdist_wheel'], '.whl',
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\build_meta.py", line 229, in _build_with_temp_dir
          self.run_setup()
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\build_meta.py", line 281, in run_setup
          super(_BuildMetaLegacyBackend,
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\build_meta.py", line 174, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 103, in <module>
          setup(
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools_dso\__init__.py", line 60, in setup
          _setup(**kws)
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\__init__.py", line 155, in setup
          return distutils.core.setup(**attrs)
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 148, in setup
          return run_commands(dist)
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 163, in run_commands
          dist.run_commands()
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 967, in run_commands
          self.run_command(cmd)
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 986, in run_command
          cmd_obj.run()
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\wheel\bdist_wheel.py", line 299, in run
          self.run_command('build')
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 986, in run_command
          cmd_obj.run()
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\_distutils\command\build.py", line 135, in run
          self.run_command(cmd_name)
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 986, in run_command
          cmd_obj.run()
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools_dso\dsocmd.py", line 509, in run
          _build_ext.run(self)
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\command\build_ext.py", line 79, in run
          _build_ext.run(self)
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 339, in run
          self.build_extensions()
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 448, in build_extensions
          self._build_extensions_serial()
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools\_distutils\command\build_ext.py", line 473, in _build_extensions_serial
          self.build_extension(ext)
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools_dso\dsocmd.py", line 512, in build_extension
          expand_sources(self, ext.sources)
        File "C:\Users\guent\AppData\Local\Temp\pip-build-env-e52ue2nf\overlay\Lib\site-packages\setuptools_dso\dsocmd.py", line 88, in expand_sources
          raise RuntimeError("Missing source file: %s"%src)
      RuntimeError: Missing source file: iocStats\devIocStats\os\default\osdCpuUsage.c
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for softioc
Failed to build softioc
ERROR: Could not build wheels for softioc, which is required to install pyproject.toml-based projects

Failed to follow "Read data from an IOC" section of tutorials

Hi all,
After creating a cothread IOC, I tried to read data in another shell. But I got the following error :

I also tried to use the epics.PV.get, and it was worked:

I don't know why, and will be very grateful for your help. Thansk a lot!

[beginner question] example/runtest not running

After building pythonSoftIoc I get the following output when trying to run runtest:

bam01:/soft/epics/inst-3.15.7/pythonIoc/example # ./runtest 
Traceback (most recent call last):
filename="../../../src/ioc/dbStatic/dbLexRoutines.c" line number=259
No such file or directory dbRead opening file ioc.db
  File "startup.py", line 10, in <module>
    softioc.devIocStats('TS-DI-TEST-01')
  File "/soft/epics/inst-3.15.7/pythonIoc/python/softioc/softioc.py", line 161, in devIocStats
    'IOCNAME=%s,name=' % ioc_name)
  File "/soft/epics/inst-3.15.7/pythonIoc/python/softioc/softioc.py", line 156, in dbLoadDatabase
    imports.dbLoadDatabase(database, path, substitutions)
  File "/soft/epics/inst-3.15.7/pythonIoc/python/softioc/imports.py", line 11, in expect_success
    assert status == 0, 'Expected success'
AssertionError: Expected success
bam01:/soft/epics/inst-3.15.7/pythonIoc/example # 

What am I doing wrong?

mbbOut on_update no longer triggers properly

As of version 4.0.0, the on_update method of an mbbOut record no longer triggers as expected - it fires once, often several seconds after a caput, and then never fires again for subsequent updates. This is true even with always_update=True. Code to recreate:

from softioc import softioc, builder, asyncio_dispatcher

dispatcher = asyncio_dispatcher.AsyncioDispatcher()

builder.SetDeviceName("ABC")

def update(new_val: int):
    print("New MBO value:", new_val)

mbo = builder.mbbOut("MBO", "A", "B", "C", "D", initial_value=0, on_update=update, always_update=True)

builder.LoadDatabase()
softioc.iocInit(dispatcher)
softioc.interactive_ioc(globals())

Then run these commands on a separate terminal:

caput("ABC:MBO", 1)
caput("ABC:MBO", 2)
caput("ABC:MBO", 3)

In my testing, using both the cothread.catools and 4p.client.cothread.Context equivalent methods, shows just a single print statement for the first caput, and none for the remaining ones. Notably, caget does return the expected value at each point, its just that the on_update method only fires once.

Note I haven't tested this for all Out record types, only for mbb records.

This was discovered while trying to update this PR to use pythonSoftIOC v4.0.0, where this issue causes a test failure: PandABlocks/PandABlocks-client#28

Allow setting of alarm status and severity on record creation using PythonSoftIOC alarm values

Currently to set the alarm status and severity on a record during creation you must use the EPICS field names and values, e.g.:

STAT="UDF", SEVR="INVALID"

It would be convenient to allow use of the same syntax that the .set_alarm() method uses, that is to say making use of the alarm enum.

This is probably somewhat related to #53.

Discussion - reccaster like module

Hi,

We have a few developers looking into using this python IOC framework and it looks really nice. All our IOCs here include IOC stats and reccaster modules and we would like to keep that going. Looking at the docs, I see devIocStats is already available.

Looks like it would be possible to hook reccaster in a similar way or have reccaster be a separate pip package. From the issue below I see there are issues with that approach. We could also write python code to do what reccaster does or even push PVs directly to CF.

#6

Just wondering if anyone is using channel finder/recsync with these IOCs and what the current thoughts are on EPICS module integration.

Thanks!

Out records without an initial_value cannot be correctly validated

If an out record is constructed without an initial_value or has no set() called before the IOC is initialised, validate methods will not work as expected - they won't work at all. An example program is:

# Import the basic framework components.
from softioc import softioc, builder, asyncio_dispatcher
import asyncio

# Create an asyncio dispatcher, the event loop is now running
dispatcher = asyncio_dispatcher.AsyncioDispatcher()

# Set the record prefix
builder.SetDeviceName("MY-DEVICE-PREFIX")

def _capture_validate(record, new_val) -> bool:
    """Check the required records have been set before allowing Capture=1"""
    try:
        raise ValueError("Test")
    except ValueError:
        return False

    return True

# Create some records
ao = builder.boolOut('BO', ZNAM="0", ONAM="1", validate=_capture_validate)

# Boilerplate get the IOC started
builder.LoadDatabase()
softioc.iocInit(dispatcher)

# Finally leave the IOC running with an interactive shell.
softioc.interactive_ioc(globals())

And running the following commands in the terminal, we see an error message BUT we also see the underlying value has changed:

[...]$ caget MY-DEVICE-PREFIX:BO
MY-DEVICE-PREFIX:BO            0
[...]$ caput MY-DEVICE-PREFIX:BO 1
Old : MY-DEVICE-PREFIX:BO            0
CA.Client.Exception...............................................
    Warning: "Channel write request failed"
    Context: "op=1, channel=MY-DEVICE-PREFIX:BO, type=DBR_STRING, count=1, ctx="MY-DEVICE-PREFIX:BO""
    Source File: ../oldChannelNotify.cpp line 160
    Current Time: Tue Oct 05 2021 09:30:02.714662686
..................................................................
New : MY-DEVICE-PREFIX:BO            1
[...]$ caget MY-DEVICE-PREFIX:BO
MY-DEVICE-PREFIX:BO            1
[eyh46967@pc0103 PandABlocks-client]$ 

Following a bit of debugging with @Araneidae it seems that all Out records will face similar issues if there is no initial_value set.

Provide stubs/annotations for RecordWrapper

Currently there's no way for a user to know what EPICS record attributes are accessible on a RecordWrapper without just trying to get the attribute - and only a small handful are accessible at the moment. This would also help static type checkers such as mypy to better reason about the values retrieved from the record.

A basic introduction to stubs can be found here: https://mypy.readthedocs.io/en/stable/stubs.html

Can't Put PV value from multiple Process

Dear All,
I try to create some PVs using pythonSoftIOC, and also create an another process to get/put the defined PVs. The code runs successfully, but the results are not corrected!(The two PV's values should be same). And I open another terminal to get the PV values, their are just the initialized ones in the builder procedure.
I don't know way, and whether it is ok using the multiple process to get/put PVs. Thanks a lot for your help and suggestions!

The code:

The result of run test code:

The result from another terminal:

Best regards,
Jiyizi

Build wheels for Python 3.10

As epicscorelibs now does

pythonIoc issues with EPICS Base 7.0.3

These are the errors I get when I run examples/runtest

Starting iocInit
############################################################################
## EPICS R7.0.3
## EPICS Base built Aug 26 2020
############################################################################
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 14
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 14
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 14
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 14
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 14
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 14
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 14
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 241, in LookupRecord
    return cls._RecordDirectory[name]
KeyError: 94101980930144
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 241, in LookupRecord
    return cls._RecordDirectory[name]
KeyError: 94101980923072
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 241, in LookupRecord
    return cls._RecordDirectory[name]
KeyError: 94101980937792
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 241, in LookupRecord
    return cls._RecordDirectory[name]
KeyError: 94101980920832
Error (514,514) PV: TS-DI-TEST-01:AO ao: init_record

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 241, in LookupRecord
    return cls._RecordDirectory[name]
KeyError: 94101980934768
Error (514,514) PV: TS-DI-TEST-01:SINP ao: init_record

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 14
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 241, in LookupRecord
    return cls._RecordDirectory[name]
KeyError: 94101980925504
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 241, in LookupRecord
    return cls._RecordDirectory[name]
KeyError: 94101980933376
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 14
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 185, in _init_record
    self = cls.LookupRecord(getattr(record, cls._link_))
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 14
Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 145, in <lambda>
    cls.__call_instance_method(method, args, record_offset)
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 157, in __call_instance_method
    self = record.DPVT
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 17
PV: TS-DI-TEST-01:STRINGIN scanAdd: I/O Intr not valid

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 145, in <lambda>
    cls.__call_instance_method(method, args, record_offset)
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 157, in __call_instance_method
    self = record.DPVT
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 17
PV: TS-DI-TEST-01:LONGSTRING scanAdd: I/O Intr not valid

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 145, in <lambda>
    cls.__call_instance_method(method, args, record_offset)
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 157, in __call_instance_method
    self = record.DPVT
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 17
PV: TS-DI-TEST-01:SIN scanAdd: I/O Intr not valid

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 145, in <lambda>
    cls.__call_instance_method(method, args, record_offset)
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 157, in __call_instance_method
    self = record.DPVT
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 17
PV: TS-DI-TEST-01:WAVEFORM scanAdd: I/O Intr not valid

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 145, in <lambda>
    cls.__call_instance_method(method, args, record_offset)
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 157, in __call_instance_method
    self = record.DPVT
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 17
PV: TS-DI-TEST-01:WAVEFORM2 scanAdd: I/O Intr not valid

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 145, in <lambda>
    cls.__call_instance_method(method, args, record_offset)
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 157, in __call_instance_method
    self = record.DPVT
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 17
PV: TS-DI-TEST-01:BOOLIN scanAdd: I/O Intr not valid

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 145, in <lambda>
    cls.__call_instance_method(method, args, record_offset)
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 157, in __call_instance_method
    self = record.DPVT
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 17
PV: TS-DI-TEST-01:LONGIN scanAdd: I/O Intr not valid

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 145, in <lambda>
    cls.__call_instance_method(method, args, record_offset)
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 157, in __call_instance_method
    self = record.DPVT
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 17
PV: TS-DI-TEST-01:AI scanAdd: I/O Intr not valid

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 145, in <lambda>
    cls.__call_instance_method(method, args, record_offset)
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 157, in __call_instance_method
    self = record.DPVT
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 17
PV: TS-DI-TEST-01:MBBI scanAdd: I/O Intr not valid

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 145, in <lambda>
    cls.__call_instance_method(method, args, record_offset)
  File "/usr/local/epics7New/pythonIoc/python/softioc/device_core.py", line 157, in __call_instance_method
    self = record.DPVT
  File "/usr/local/epics7New/pythonIoc/python/softioc/fields.py", line 153, in __getattr__
    ctypes_type = DbfCodeToCtypes[field_type]
KeyError: 17
iocRun: All initialization complete
Python 3.6.9 (default, Jul 17 2020, 12:50:27) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)

Naming convention inconsistency

There's a small naming convention inconsistency with the names of the record types: Action, WaveformIn and WaveformOut are all PascalCase, and every other record is camelCase (aOut, mbbIn, etc.).

On a related note, WaveformIn is defined slightly differently to WaveformOut - it's actually "Waveform" - which causes __name__ to show an unexpected string, and IDE intellisense to highlight it differently

This naming convention has been around for some time, and so changing it would be potentially disruptive.

pythonIoc using a conda environment

This is what I did to use a conda environment with pythonIoc:

1. modify configure/CONFIG_SITE:

PYTHON = /operation/common/miniconda3/envs/mbf/bin/python3.7
PYTHON_CONFIG = $(PYTHON)-config

2. activate the conda environment
3. install python-config:

pip install python-config

4. call make
   The first strange thing here is that it doesn't work if the conda environment is not activated, which is not what I was expecting since the path to python3.7 is given.
   But OK, the conda environment has to be activated, let's continue!
5. start the IOC
   Then I get the following error:

/mntdirect_host/_operation_ro/control/os/debian9/epics/components/pythonIoc-py37/bin/linux-x86_64/softIoc: error while loading shared libraries: libpython3.7m.so.1.0: cannot open shared object file: No such file or directory

After adding the folder /operation/common/miniconda3/envs/mbf/lib in LD_LIBRARY_PATH it's OK.
Activating or not the conda environment doesn't make any difference: the correct path to python is used (contrary to step 4.).

So I was wondering if there is a way to link softIoc with the correct path of libpython3.7m.so.1.0?

edit: Following #1 I tryied to set PYTHONHOME=/operation/common/miniconda3/envs/mbf, but it has no effect.

Waveform records ignore NELM keyword if initial_value is set

If you specify both an initial_value=... and NELM=... keywords when creating a WaveformOut record, the NELM keyword is ignored and the waveform is created with a maximum length equal to the length of the data.

I think the offending line is L169 of builder.py, which seems to override the passed in NELM without checking if it is set:
fields['NELM'] = length

There is a workaround: instead of initial_value and NELM, use something like this (assuming data is a numpy array)

length=required_length,
datatype=data.dtype,

And later do myrecord.set(data)

No protection against overflow/underflow when setting values

pythonSoftIOC provides no protections against overflowing data type limits.

The most obvious example of this is longIn/Out, due to the unbounded size of Python3's int type. No warning or error is raised if you attempt to do the following:

builder.longOut("TEST", initial_value=9999999999999)

[...]$ caget TEST
PREFIX:TEST 1316134911

The value is truncated, but no warning is given.

This issue should also investigate all other types, and consider adding validation to the incoming values before passing to EPICS.
Possibilities include ( but are not limited to):

• Huge floating point values
• Tiny floating point values
• Strings that may be silently truncated, especially when unicode is in use
• Waveforms when the value being set is longer than the NELM of the record

pythonSoftIOC won't install on RHEL8

The python version in RHEL8 is 3.6.8 which is not allowed in setup.cfg, I will try to install in a local virtual environment to check what's the compatibility issue

An error

Hi,
I am trying to make my own motorRecords by modifying db and found examples under tests.
My softioc is pip installed, and it produces the error below:

pythonSoftIOC/tests] python sim_asyncio_ioc_override.py myprefix
Traceback (most recent call last):
File "sim_asyncio_ioc_override.py", line 36, in
softioc.iocInit(asyncio_dispatcher.AsyncioDispatcher(event_loop))
TypeError: init() takes 1 positional argument but 2 were given

Would be nice to separate out devIocStats

In #5 we build devIocStats into the extension module as builder.py calls registerRecordDeviceDrivers which needs all .so files to be loaded beforehand. It would be nice to separate this out into another Python module, but this would be a backwards incompatible change