get_sensor_reading() Returns 'CompletionCodeError cc=0xcc desc=Invalid data field in Request'

[sbwebb]

I am trying to use this ipmi library to read sdr and sensor data from VadaTech devices. Currently I can read data using ipmitool and freeipmi library. And I can create a session to the device following your online example below. However, I cannot read any sensors or LEDs. I assume that it has something to do with bridging/routing, but I do not know since ipmitool does not require any bridging info. But freeipmi on the other hand does require this switch, '--bridge-sensors'

interface = pyipmi.interfaces.create_interface(interface='rmcp',
slave_address=0x81,
host_target_address=0x20,
keep_alive_interval=1)
ipmi = pyipmi.create_connection(interface)
ipmi.session.set_session_type_rmcp(host='192.168.201.140', port=623)
ipmi.session.set_auth_type_user(username='', password='')

ipmi.target = pyipmi.Target(ipmb_address=0x82)

ipmi.session.establish()
device_id = ipmi.get_device_id()

[sbwebb]

Additionally, here is an example of using iptmitool and then pyipmi to read the same sensor.

ipmitool -I lan -H 192.168.201.140 -U "" -P "" -t 0x82 -l 0x1 raw 0x04 0x2d 0x02
1c c0 c0 00

Code:

interface = pyipmi.interfaces.create_interface(interface='rmcp',
slave_address=0x81,
host_target_address=0x20,
keep_alive_interval=1)
ipmi = pyipmi.create_connection(interface)
ipmi.session.set_session_type_rmcp(host='192.168.201.140', port=623)
ipmi.session.set_auth_type_user(username='', password='')
ipmi.target = pyipmi.Target(ipmb_address=0x82)
ipmi.session.establish()
print(ipmi.get_sensor_reading(sensor_number=0x02, lun=1))

Output:

[test]$ python3 ipmix.py
Traceback (most recent call last):
File "ipmix.py", line 28, in
print(ipmi.get_sensor_reading(sensor_number=0x02, lun=1))
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/sensor.py", line 173, in get_sensor_reading
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/init.py", line 213, in send_message_with_name
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/utils.py", line 76, in check_rsp_completion_code
pyipmi.errors.CompletionCodeError: CompletionCodeError cc=0xcc desc=Invalid data field in Request

[sbwebb]

But then seems work when I change interface to 'ipmitool'. So, how am I supposed to use this tool? I thought 'rmcp' was the way to go?

My intent is to build this in a threaded application to monitor about 100 VadaTech chassis. Can someone in this community help me out with the library's use?

Code:
interface = pyipmi.interfaces.create_interface(interface='ipmitool',
interface_type='lan')
ipmi = pyipmi.create_connection(interface)
ipmi.session.set_session_type_rmcp(host='192.168.201.140', port=623)
ipmi.session.set_auth_type_user(username='', password='')
ipmi.target = pyipmi.Target(ipmb_address=0x82)
ipmi.session.establish()
print(ipmi.get_sensor_reading(sensor_number=0x02, lun=1))

Output:
[test]$ python3 ipmix.py
(28, 192)

[sbwebb]

Meanwhile, back to trying the RMCP interface, I'm getting closer to getting it to work. But even though I can see in wireshark that my vadatech device returns the sensor reading to my request, pyimpi throws error while decoding it.

See code below, see req/rsp in wireshark below. Some help would be much appreciated.

Error:
Traceback (most recent call last):
File "ipmix.py", line 34, in
print(ipmi.get_sensor_reading(sensor_number=2, lun=1))
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/sensor.py", line 173, in get_sensor_reading
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/init.py", line 212, in send_message_with_name
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/init.py", line 196, in send_message
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/interfaces/rmcp.py", line 662, in send_and_receive
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/interfaces/rmcp.py", line 620, in _send_and_receive
pyipmi.errors.RetryError: Max retry while checking receiveddata against request header for rmcphost 192.168.201.140

Code:

interface = pyipmi.interfaces.create_interface(interface='rmcp',
                                         slave_address=0x81,
                                         host_target_address=0x20,
                                         keep_alive_interval=1)

ipmi = pyipmi.create_connection(interface)
ipmi.session.set_session_type_rmcp(host='192.168.201.140', port=623)
ipmi.session.set_auth_type_user(username='', password='')
ipmi.session.set_priv_level('ADMINISTRATOR')

ipmi.target = pyipmi.Target(ipmb_address=0x20, routing=[(0x81, 0x20, 0), (0x20, 0x82, None)])
ipmi.session.establish()

print(ipmi.get_sensor_reading(sensor_number=2, lun=1))

[sbwebb]

@hthiery ?

[hthiery]

Did you try to decode the response message? In this example you used a bridge message with SendMsg. Do you realy need to do that?

How is the BMC/IPMC instrastructure of the devices you want to query?

[hthiery]

It looks like the rs_lun does not match https://github.com/kontron/python-ipmi/blob/master/pyipmi/interfaces/ipmb.py#L241

rs_sa=0x82, rs_lun=0x01, rq_sa=0x20, rq_lun=0, rq_seq=5, netfn=0x04, cmdid=0x2d
rs_sa=0x20, rs_lun=0x00, rq_sa=0x81, rq_lun=0, rq_seq=5, netfn=0x05, cmdid=0x2d

[sbwebb]

First, thank you @hthiery for taking the time to reply.

Did you try to decode the response message?
Yes. I was trying. I was trying to compare the structure to what is outlined in the IPMI spec, section 13.8.

In this example you used a bridge message with SendMsg. Do you realy need to do that?
I really am not sure if I need to do that or not. All I can go by is what I have done with other tools. And the hardware documentation specifies that the sensor owner is the carrier manager, 0x82

For example, in ipmitool:
ipmitool -I lan -H 192.168.201.140 -p 623 -L ADMINISTRATOR -U "" -P "" -t 0x82 -l 1 raw 0x04 0x2d 0x02
1c c0 c0 00

Wireshark output from command above:
0000 00 13 3a 0d 42 02 e0 70 ea da 10 ee 08 00 45 00
0010 00 3a a4 aa 40 00 40 11 81 73 c0 a8 c9 b7 c0 a8
0020 c9 8c fd ef 02 6f 00 26 14 cd 06 00 ff 07 00 07
0030 00 00 00 68 00 00 00 10 20 18 c8 81 28 34 40 82
0040 11 6d 20 28 2d 02 89 e3

How is the BMC/IPMC instrastructure of the devices you want to query?
I am not sure how to answer this. All I know is that 0x82 is the carrier manager and 0xA8 is the slave address of the cooling unit that I am trying to read the sensor of. and that the ipmitool command above could be also written as the one below where we specify the cooling unit's slave address
ipmitool -I lan -H 192.168.201.140 -U "" -P "" -T 0x82 -t 0xa8 raw 0x04 0x2d 0x10
1c c0 c0 00

It looks like the rs_lun does not match https://github.com/kontron/python-ipmi/blob/master/pyipmi/interfaces/ipmb.py#L241
rs_sa=0x82, rs_lun=0x01, rq_sa=0x20, rq_lun=0, rq_seq=5, netfn=0x04, cmdid=0x2d
rs_sa=0x20, rs_lun=0x00, rq_sa=0x81, rq_lun=0, rq_seq=5, netfn=0x05, cmdid=0x2d

So if this is true, which I can see that it is by decoding the netFn/rsLUN for the req/rsp then why are the other tools not choking on this? And what now? Reach out to the vendor and tell them that there device is responding to request incorrectly?

[hthiery]

First, thank you @hthiery for taking the time to reply.

Did you try to decode the response message?
Yes. I was trying. I was trying to compare the structure to what is outlined in the IPMI spec, section 13.8.

In this example you used a bridge message with SendMsg. Do you realy need to do that?
I really am not sure if I need to do that or not. All I can go by is what I have done with other tools. And the hardware documentation specifies that the sensor owner is the carrier manager, 0x82

For example, in ipmitool: ipmitool -I lan -H 192.168.201.140 -p 623 -L ADMINISTRATOR -U "" -P "" -t 0x82 -l 1 raw 0x04 0x2d 0x02 1c c0 c0 00

Wireshark output from command above: 0000 00 13 3a 0d 42 02 e0 70 ea da 10 ee 08 00 45 00 0010 00 3a a4 aa 40 00 40 11 81 73 c0 a8 c9 b7 c0 a8 0020 c9 8c fd ef 02 6f 00 26 14 cd 06 00 ff 07 00 07 0030 00 00 00 68 00 00 00 10 20 18 c8 81 28 34 40 82 0040 11 6d 20 28 2d 02 89 e3

Ok .. it looks that the carrier manager "merges" the sensors of the connected devices to an own representation. Then it makes sense that you need to specify the lun for the request. In this case it is exepected that the carrier manager answers with the correct lun. Seems to be problem from the implementation. You could test if in case you remove the lun check from the rx_filter if it works.

Another option is to setup the correct bridged message (routing information) to directly query the device you want (power module). Here the carrier manager is responsible to transfer the request to the specified bridged device and returns the unmodified message from the device (power module). in this case the lun should be 0.

How is the BMC/IPMC instrastructure of the devices you want to query?
I am not sure how to answer this. All I know is that 0x82 is the carrier manager and 0xA8 is the slave address of the cooling unit that I am trying to read the sensor of. and that the ipmitool command above could be also written as the one below where we specify the cooling unit's slave address
ipmitool -I lan -H 192.168.201.140 -U "" -P "" -T 0x82 -t 0xa8 raw 0x04 0x2d 0x10
1c c0 c0 00

It looks like the rs_lun does not match https://github.com/kontron/python-ipmi/blob/master/pyipmi/interfaces/ipmb.py#L241
rs_sa=0x82, rs_lun=0x01, rq_sa=0x20, rq_lun=0, rq_seq=5, netfn=0x04, cmdid=0x2d
rs_sa=0x20, rs_lun=0x00, rq_sa=0x81, rq_lun=0, rq_seq=5, netfn=0x05, cmdid=0x2d

So if this is true, which I can see that it is by decoding the netFn/rsLUN for the req/rsp then why are the other tools not choking on this? And what now? Reach out to the vendor and tell them that there device is responding to request incorrectly?

[hthiery]

you can have a look at for an example: https://github.com/kontron/python-ipmi/blob/master/pyipmi/__init__.py#L132

[sbwebb]

The routing is very hard to figure out. I can see a difference in the hex bytes but I can't find anything in the ipmi spec about how these routed messages get built.

ipmitool: ipmitool -I lan -H 192.168.201.140 -U "" -P "" -T 0x82 -t 0xa8 raw 0x04 0x2d 0x10
0000 00 13 3a 0d 42 02 e0 70 ea da 10 ee 08 00 45 00 ..:.B.àpêÚ.î..E.
0010 00 42 86 be 40 00 40 11 9f 57 c0 a8 c9 b7 c0 a8 .B.¾@.@..WÀ¨É·À¨
0020 c9 8c 9c 26 02 6f 00 2e 14 d5 06 00 ff 07 00 07 É..&.o...Õ..ÿ...
0030 00 00 00 02 00 00 00 18 20 18 c8 81 28 34 40 82 ........ .È.(4@.
0040 18 66 20 28 34 40 a8 10 48 20 28 2d 10 7b 44 e3 .f (4@¨.H (-.{Dã

ipmi.target = pyipmi.Target(ipmb_address=0xa8, routing=[(0x81, 0x20, 0), (0x20, 0x82, None), (0x20, 0xa8, None)])
print(ipmi.get_sensor_reading(sensor_number=0x10, lun=0x00))

pyipmi:
0000 00 13 3a 0d 42 02 e0 70 ea da 10 ee 08 00 45 00 ..:.B.àpêÚ.î..E.
0010 00 52 dc ae 40 00 40 11 49 57 c0 a8 c9 b7 c0 a8 .RÜ®@.@.IWÀ¨É·À¨
0020 c9 8c e3 43 02 6f 00 3e 14 e5 06 00 ff 07 02 03 É.ãC.o.>.å..ÿ...
0030 00 00 00 04 00 00 00 b0 67 fd 64 68 32 cb e0 21 .......°gýdh2Ëà!
0040 ac e3 46 c1 7b 46 eb 18 20 18 c8 81 14 34 40 82 ¬ãFÁ{Fë. .È..4@.
0050 18 66 20 14 34 40 a8 10 48 20 14 2d 10 8f 58 f7 .f .4@¨.H .-..X÷

Traceback (most recent call last):
File "ipmix.py", line 35, in
print(ipmi.get_sensor_reading(sensor_number=0x10, lun=0x00))
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/sensor.py", line 173, in get_sensor_reading
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/init.py", line 212, in send_message_with_name
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/init.py", line 196, in send_message
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/interfaces/rmcp.py", line 662, in send_and_receive
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/interfaces/rmcp.py", line 620, in _send_and_receive
pyipmi.errors.RetryError: Max retry while checking receiveddata against request header for rmcphost 192.168.201.140

[hthiery]

try to use the ipmitool interface type and check again please

[sbwebb]

Ok. at first I got this error:
python3 ipmix.py
Traceback (most recent call last):
File "ipmix.py", line 35, in
print(ipmi.get_sensor_reading(sensor_number=0x10, lun=0x00))
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/sensor.py", line 173, in get_sensor_reading
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/init.py", line 212, in send_message_with_name
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/init.py", line 196, in send_message
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/interfaces/ipmitool.py", line 174, in send_and_receive
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/interfaces/ipmitool.py", line 137, in send_and_receive_raw
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/interfaces/ipmitool.py", line 245, in _build_ipmitool_cmd
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/interfaces/ipmitool.py", line 206, in _build_ipmitool_target
TypeError: %d format: a number is required, not NoneType

But then I changed a None to a zero in the middle tuple and it worked.
ipmi.target = pyipmi.Target(ipmb_address=0xa8, routing=[(0x81, 0x20, 0), (0x20, 0x82, 0), (0x20, 0xa8, None)])
$ python3 ipmix.py
(28, 192)

But then I switched back to the rmcp interface and it did not work:
$ python3 ipmix.py
Traceback (most recent call last):
File "ipmix.py", line 36, in
print(ipmi.get_sensor_reading(sensor_number=0x10, lun=0x00))
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/sensor.py", line 173, in get_sensor_reading
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/init.py", line 212, in send_message_with_name
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/init.py", line 196, in send_message
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/interfaces/rmcp.py", line 662, in send_and_receive
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/interfaces/rmcp.py", line 620, in _send_and_receive
pyipmi.errors.RetryError: Max retry while checking receiveddata against request header for rmcphost 192.168.201.140

[sbwebb]

Looks to me like the difference between a message created with ipmitool vs pyipmi is a 0x28 and 0x14 in the message.

ipmitool sends this:
20 18 c8 81 28 34 40 82 18 66 20 28 34 40 a8 10 48 20 28 2d 10 7b 44 e3

And we receive this:
81 1c 63 20 28 34 00 20 14 cc a8 20 2d 00 1c c0 c0 00 6f 84

While pyipmi sends this:
20 18 c8 81 14 34 40 82 18 66 20 14 34 40 a8 10 48 20 14 2d 10 8f 58 f7

But the device ACTUALLY responds with this:
81 1c 63 20 14 34 00 20 14 cc a8 d8 2d 00 1c c0 c0 00 b7 98

And pyimpi complains here:
python3 ipmix.py
Traceback (most recent call last):
File "ipmix.py", line 36, in
print(ipmi.get_sensor_reading(sensor_number=0x10))
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/sensor.py", line 173, in get_sensor_reading
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/init.py", line 212, in send_message_with_name
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/init.py", line 196, in send_message
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/interfaces/rmcp.py", line 662, in send_and_receive
File "/usr/local/lib/python3.6/site-packages/python_ipmi-unknown-py3.6.egg/pyipmi/interfaces/rmcp.py", line 620, in _send_and_receive
pyipmi.errors.RetryError: Max retry while checking receiveddata against request header for rmcphost 192.168.201.140

So what does the 5th octet mean? If it still means netFn/LUN then those definitely are not matching up between req/rsp

[hthiery]

Ah I see .. the response message from the device is encapsulated in a the send message response. This encapsulated response is currently not supported by pyipmi.

There are 2 options to implement send message

1. received response to send message without encapsualted response and the "real" response as separate message (implemented in pyipmi)
2. response with the encapsulated response (not supported in pyipmi)

you could try to add this support in pyipmi like it is done e.g. in ipmitool and fallback to the ipmitool interface type supported by pyipmi.

[sbwebb]

Ok. I understand what you are saying. Is the encapsulated response something that is left up to the vendors/implementors to decide if they should do it or not? Like the ipmi specification leaves that part optional?