hal: Add halcompupdate to migrate .comp files to the new HAL API

Out-of-tree .comp components using the legacy HAL types (float, bit,
s32, u32, s64, u64, signed, unsigned) and direct pin/param assignment
stop working when the HAL API break is performed (#4099, #4247).
halcompupdate rewrites them to the new API automatically:

* declaration types are converted: float->real, bit->bool, s32->si32,
  u32->ui32, s64->sint, u64->uint, signed->si32, unsigned->ui32
  ('port' is left alone, it has no new-style replacement yet)
* writes to out/io pins and to params become <name>_set(...) calls,
  including compound assignments, ++/--, array pins, chained
  assignments, *<name>_ptr dereferences and writes inside #define
  macros (macro parameters shadow same-named pins)
* legacy C types are modernized (double/real_t -> rtapi_real,
  hal_bit_t -> volatile rtapi_bool, ...)
* reads are unchanged and pins are never renamed, so existing HAL
  configurations keep working

Constructs that cannot be converted safely are left unchanged with a
warning for manual conversion: taking the address of a pin/param,
direct use of the legacy hal_pin_*_new/hal_param_*_new creation API,
postfix ++/-- whose value is used, and array indices with side
effects.

In-place rewriting is atomic (temp file + rename) and keeps a .bak
backup created with O_EXCL|O_NOFOLLOW.

halcompile now warns once per deprecated type, pointing to
halcompupdate(1), at the spot previously marked for this warning.
Docs: migration section in comp.adoc, new halcompupdate(1) manpage,
SEE ALSO in halcompile(1).  Regression test in
tests/halcompile/update-api.

Validated by converting all in-tree components from master: 119/124
compile (the other 5 need in-tree headers and fail identically for
the already-converted versions), and the output matches the hand
conversions in #4247 functionally.

Author: grandixximo

debian/linuxcnc-uspace-dev.install

@@ -1,4 +1,5 @@
 usr/bin/halcompile
+usr/bin/halcompupdate
 usr/bin/modcompile
 usr/include/linuxcnc
 usr/lib/liblinuxcnc.a

debian/linuxcnc-uspace-dev.manpages

@@ -1,2 +1,3 @@
 usr/share/man/man1/halcompile.1
+usr/share/man/man1/halcompupdate.1
 usr/share/man/man3/*

docs/src/hal/comp.adoc

@@ -267,8 +267,14 @@ In this case, r-strings are particularly useful, because the backslashes in an r
 r"\fIexample\fB"
 ----
 
-* 'TYPE' - One of the HAL types: 'bit', 's32', 'u32', 's64', 'u64' or 'float'.
-  The names 'signed' and 'unsigned' may also be used for 's32' and 'u32' but 's32' and 'u32' are preferred.
+* 'TYPE' - One of the HAL types: 'real', 'bool', 'si32', 'ui32', 'sint' or 'uint'.
+  'real' is a floating point value, 'bool' a boolean, 'si32' and 'ui32' are 32-bit signed and unsigned integers,
+  and 'sint' and 'uint' are 64-bit signed and unsigned integers.
+  The legacy types 'float' (for 'real'), 'bit' (for 'bool'), 's32' (for 'si32'), 'u32' (for 'ui32'),
+  's64' (for 'sint') and 'u64' (for 'uint') are still accepted, as are the aliases 'signed' and 'unsigned'
+  (for 'si32' and 'ui32'), but they are deprecated, 'halcompile' will warn about them,
+  and they will be removed when the HAL API break is performed.
+  See <<sub:hal-comp-migration,Migrating to the new HAL API>>.
 * 'PINDIRECTION' - One of the following: 'in', 'out', or 'io'.
   A component sets a value for an 'out' pin, it reads a value from an 'in' pin, and it may read or set the value of an 'io' pin.
 * 'PARAMDIRECTION' - One of the following: 'r' or 'rw'. A component sets a value for a 'r' parameter, and it may read or set the value of a 'rw' parameter.
@@ -458,6 +464,15 @@ The details of `struct __comp_state` and these macros may change from one versio
 +
 When the item is a conditional item, it is only legal to refer to it when its 'condition' evaluated to a nonzero value.
 
+* `pin_name_set(`__value__`)` or `param_name_set(`__value__`)` - For each 'out' or 'io' pin and each parameter
+  of a new-style type ('real', 'bool', 'si32', 'ui32', 'sint', 'uint'), there is a macro which sets the value of the pin or parameter.
+  For arrays the form is 'pin_name_set(idx, value)'.
+  With the new-style types, assigning to an 'out' or 'io' pin or to a parameter with plain C assignment is not possible;
+  the '_set' macro must be used instead (reading stays transparent through the bare name).
+  The '_set' macro evaluates to the value that was set, so chained use like 'a_set(b_set(x))' works.
+  There is also a 'pin_name_ptr' macro, but with new-style types it evaluates to an opaque reference
+  which can only be used with the 'hal_get_*' and 'hal_set_*' functions; it cannot be dereferenced.
+
 * 'variable_name' - For each variable 'variable_name' there is a macro which allows the name to be used on its own to refer to the variable.
   When 'variable_name' is an array, the normal C-style subscript is used: 'variable_name[idx]'.
 * 'data' - If "option data" is specified, this macro allows access to the instance data.
@@ -466,6 +481,47 @@ When the item is a conditional item, it is only legal to refer to it when its 'c
   This macro iterates over all the defined instances.
   Inside the body of the loop, the 'pin_name', 'parameter_name', and 'data' macros work as they do in realtime functions.
 
+[[sub:hal-comp-migration]]
+== Migrating to the new HAL API
+
+HAL is moving from direct memory access of pin and parameter values to strongly typed getter/setter access.
+New-style declaration types replace the legacy types:
+
+[cols="1,1,3",options="header"]
+|===
+| Legacy type | New type | Underlying C type
+| 'float'     | 'real'   | 'rtapi_real' (floating point)
+| 'bit'       | 'bool'   | 'rtapi_bool' (boolean)
+| 's32'       | 'si32'   | 'rtapi_s32' (32-bit signed)
+| 'u32'       | 'ui32'   | 'rtapi_u32' (32-bit unsigned)
+| 's64'       | 'sint'   | 'rtapi_sint' (64-bit signed)
+| 'u64'       | 'uint'   | 'rtapi_uint' (64-bit unsigned)
+| 'signed'    | 'si32'   | 'rtapi_s32' (32-bit signed)
+| 'unsigned'  | 'ui32'   | 'rtapi_u32' (32-bit unsigned)
+|===
+
+The legacy types still work, but 'halcompile' warns about them and they will be removed when the HAL API break is performed.
+With the new-style types:
+
+* reading a pin or parameter is unchanged (the bare name, or 'name(idx)' for arrays, evaluates to the value);
+* writing an 'out' or 'io' pin or a parameter requires the generated 'name_set(value)' macro
+  (or 'name_set(idx, value)' for arrays) instead of plain assignment;
+* 'name_ptr' is an opaque reference for use with 'hal_get_*'/'hal_set_*' and can no longer be dereferenced.
+
+Existing components can be migrated automatically with the 'halcompupdate' tool:
+
+----
+halcompupdate mycomponent.comp          # show a diff of the required changes
+halcompupdate -i mycomponent.comp       # rewrite the file in place (keeps a .bak)
+----
+
+The tool converts the declaration types and rewrites writes to pins and parameters
+(including compound assignments and dereferences of 'name_ptr') to the '_set' form.
+It deliberately does not rename pins or parameters, because their names are part of the HAL configuration interface.
+Components that use the legacy 'hal_pin_*_new'/'hal_param_*_new' creation functions or that pass
+the address of a pin ('&name') to helper functions cannot be converted automatically;
+'halcompupdate' prints a warning for those cases so they can be converted by hand.
+
 == Components with one function
 
 If a component has only one function and the string `"FUNCTION"` does not appear anywhere after `;;`,

docs/src/man/man1/halcompile.1.adoc

@@ -99,6 +99,9 @@ Extra arguments passed to the linker.
 
 == SEE ALSO
 
+* *halcompupdate*(1) to migrate existing *.comp* files to the new HAL
+  pin/param API (new-style declaration types and *<name>_set()* accessors)
+
 * _Halcompile_ / _HAL Component Generator_ in the LinuxCNC documentation for a
   full description of the *.comp* syntax, along with examples
 

docs/src/man/man1/halcompupdate.1.adoc

@@ -0,0 +1,98 @@
+= halcompupdate(1)
+
+== NAME
+
+halcompupdate - Migrate HAL .comp components to the new HAL pin/param API
+
+== SYNOPSIS
+
+*halcompupdate* [--in-place] [--no-backup] [--check] [--no-c-types] [--quiet] compfile...
+
+== DESCRIPTION
+
+*halcompupdate* converts HAL components written for the legacy HAL API
+(pins and params declared with the types *float*, *bit*, *s32*, *u32*,
+*s64*, *u64*, *signed* or *unsigned* and written with plain C assignment)
+to the new HAL API, where pins and params are declared with the types
+*real*, *bool*, *si32*, *ui32*, *sint* and *uint* and written with the
+generated *<name>_set(value)* accessor.
+
+The legacy types still work, but halcompile(1) warns about them and they
+will be removed when the HAL API break is performed.
+
+The following transformations are applied:
+
+* Declaration types are replaced: *float* -> *real*, *bit* -> *bool*,
+  *s32* -> *si32*, *u32* -> *ui32*, *s64* -> *sint*, *u64* -> *uint*,
+  *signed* -> *si32*, *unsigned* -> *ui32*.  The type *port* is not
+  converted yet.
+* Assignments to *out* and *io* pins and to parameters are rewritten to
+  the *<name>_set(...)* form, including compound assignments
+  (*name += x* becomes *name_set(name + (x))*), increments/decrements
+  and array pins (*name(i) = x* becomes *name_set(i, x)*).
+* Dereferences of the old pin pointer macro are rewritten:
+  ***name_ptr*** reads become *(name)* and ***name_ptr = x*** writes
+  become *name_set(x)*.
+* Legacy C types are modernized with their variable-use replacements
+  (*double* -> *rtapi_real*, *hal_float_t* -> *rtapi_real*,
+  *hal_bit_t* -> *rtapi_bool*, and so on).  The *volatile* qualifier of
+  the legacy types is dropped on purpose: it existed for direct access
+  to HAL memory and was never right for variables.  Pointers to the
+  legacy types referenced HAL memory and become opaque references in the
+  new API; those are warned about and left unchanged.  Use
+  *--no-c-types* to skip this conversion.
+
+Reading pins and params is unchanged: the bare name (or *name(idx)* for
+arrays) evaluates to the value in both APIs.
+
+Pin and parameter names are never changed, because they are part of the
+HAL configuration interface used by .hal files.
+
+Components that use the legacy *hal_pin_*_new*/*hal_param_*_new* creation
+functions directly, or that pass the address of a pin (*&name*) to helper
+functions, cannot be converted automatically.  *halcompupdate* prints a
+warning for those cases so they can be converted by hand.
+
+== OPTIONS
+
+*compfile...*::
+One or more .comp files to convert.
+Without other options, a unified diff of the required changes is printed.
+
+*-i*, *--in-place*::
+Rewrite the files in place.  A backup with the suffix *.bak* is kept
+unless *--no-backup* is given.
+
+*--no-backup*::
+With *--in-place*, do not keep a *.bak* backup.
+
+*--check*::
+Do not write anything; exit with status 1 if any file would be changed.
+Useful in build systems to verify that components are migrated.
+
+*--no-c-types*::
+Only convert pin/param declarations and accesses; do not modernize
+C types in the component body.
+
+*-q*, *--quiet*::
+Suppress warnings on stderr.
+
+== EXAMPLES
+
+Show what would change:
+
+----
+halcompupdate mycomponent.comp
+----
+
+Migrate in place:
+
+----
+halcompupdate -i mycomponent.comp
+----
+
+== SEE ALSO
+
+*halcompile*(1), the _Halcompile_ / _HAL Component Generator_ section in
+the LinuxCNC documentation for a full description of the *.comp* syntax
+and the new HAL API.

src/hal/utils/Submakefile

@@ -96,6 +96,12 @@ endif
 	$(ECHO) Copying python script $(notdir $@)
 	$(Q)(echo '#!$(PYTHON)'; sed '1 { /^#!/d; }' $<) > $@.tmp && chmod +x $@.tmp && mv -f $@.tmp $@
 
+../bin/halcompupdate: ../bin/%: hal/utils/%.py
+	@$(ECHO) Syntax checking python script $(notdir $@)
+	$(Q)$(PYTHON) -m py_compile $<
+	$(ECHO) Copying python script $(notdir $@)
+	$(Q)(echo '#!$(PYTHON)'; sed '1 { /^#!/d; }' $<) > $@.tmp && chmod +x $@.tmp && mv -f $@.tmp $@
+
 ../bin/modcompile: ../bin/%: hal/drivers/mesa-hostmot2/modbus/%.py
 	@$(ECHO) Syntax checking python script $(notdir $@)
 	$(Q)$(PYTHON) -m py_compile $<
@@ -111,7 +117,7 @@ endif
 	$(ECHO) Copying python script $(notdir $@)
 	$(Q)(echo '#!$(PYTHON)'; sed '1 { /^#!/d; }' $<) > $@.tmp && chmod +x $@.tmp && mv -f $@.tmp $@
 
-TARGETS += ../bin/halcompile ../bin/elbpcom ../bin/modcompile ../share/linuxcnc/mesa_modbus.c.tmpl ../bin/mesambccc
+TARGETS += ../bin/halcompile ../bin/elbpcom ../bin/modcompile ../share/linuxcnc/mesa_modbus.c.tmpl ../bin/mesambccc ../bin/halcompupdate
 objects/%.py: %.g
 	@mkdir -p $(dir $@)
 	$(Q)$(YAPPS) $< $@

src/hal/utils/halcompile.g

@@ -151,9 +151,11 @@ def parse(filename):
 
 dirmap = {'r': 'HAL_RO', 'rw': 'HAL_RW', 'in': 'HAL_IN', 'out': 'HAL_OUT', 'io': 'HAL_IO' }
 typemap = {'signed': 's32', 'unsigned': 'u32'}
-deprmap = {'s32': 'signed', 'u32': 'unsigned'}
-deprecated = ['s32', 'u32']
 newtypes = ['bool', 'sint', 'uint', 'si32', 'ui32', 'real']
+# Old HAL declaration types and their new-style replacements.  These types
+# will be removed when the HAL API break is performed.
+old2newtypes = {'float': 'real', 'bit': 'bool', 's32': 'si32', 'u32': 'ui32',
+                's64': 'sint', 'u64': 'uint', 'signed': 'si32', 'unsigned': 'ui32'}
 
 def initialize():
     global functions, params, pins, comp_name, names, docs, variables
@@ -162,6 +164,7 @@ def initialize():
     functions = []; params = []; pins = []; options = {}; variables = []
     modparams = []; docs = []; includes = [];
     comp_name = None
+    deprecated_type_warnings.clear()
 
     names = {}
 
@@ -207,8 +210,14 @@ def see_also(doc):
 def notes(doc):
     docs.append(('notes', doc));
 
+deprecated_type_warnings = set()
+
 def type2type(type_):
-    # When we start warning about s32/u32 this is where the warning goes
+    if type_ in old2newtypes and type_ not in deprecated_type_warnings:
+        deprecated_type_warnings.add(type_)
+        Warn("HAL type '%s' is deprecated and will be removed at the HAL API "
+             "break; use '%s' instead. Run halcompupdate(1) to migrate "
+             "automatically." % (type_, old2newtypes[type_]))
     return typemap.get(type_, type_)
 
 def checkarray(name, array):