| Author |
 Topic  |
|
|
ullix
Germany
1168 Posts |
 Posted - 01/31/2017 : 08:56:02
|
My device is "GQ GMC-300E plus" with firmware "GMC-300Re 4.20". I access the device with my Python script under Linux Ubuntu and try to read histroical data with the SPIR command.
The GQ-RFC1201, Geiger Counter Communication Protocol document says:
quote:
The length normally not exceed 4096 bytes in each request.
When I ask for 4095 (1 less) bytes beginning at the address 0x00 I get them all (from #0 to #4094) in under 1 second. But when I ask for 4096 the device stalls. When opening the serial port with a 10sec timeout I get a single byte only. Same failure when I try to read anything exceeding 4096 bytes.
When I use an address of 256 (instead of zero) and again read only 4095 bytes it again works, demonstrating that I can read beyond 4096 bytes (though last 600 or so bytes are all 0xff), and fails when trying 4096 bytes.
So, can I actually read 4096 (0x10 0x00) bytes or is 4095 (0x0f 0xff) the maximum chunk?
|
|
| Reply #1
phgphd
USA
12 Posts |
Posted - 02/02/2017 : 17:06:11
|
| The history buffer on the GMC is 64K bytes deep. So reading starting at address 256 reads bytes from 256 to 256+4095 and will be successful. As to why only 4095 can be read successfully, I can only speculate that the GMC firmware has a bug. A work around would be to read 2k bytes at a time which only doubles the amount of reads necessary to get the whole 64K bytes. |
 |
|
| Reply #2
ZLM
1261 Posts |
Posted - 02/02/2017 : 19:39:34
|
The address start from 0. So, your 4096 address is 4097 bytes long. This will cause the firmware into a dead loop.
The protocol said not exceed the 4096 bytes which is the address from 0 to 4095.
See below commands:
SPIR 00 00 00 0F FF read first 4K history data.
SPIR 00 10 00 0F FF read second 4K history data.
SPIR 00 20 00 0F FF read third 4K history data.
.....
SPIR 00 F0 00 0F FF read last 4K history data from GMC-300.
....
SPIR 0F 00 00 0F FF read last 4K history data from GMC-320. |
|
|
| Reply #3
ullix
Germany
1168 Posts |
Posted - 02/05/2017 : 05:22:25
|
I ended using 2048 bytes as a chunk for a single SPIR command, and this works and works fast enough.
However, it is not true that I am reading 4097 bytes when requesting 4096. See this outcome from the code pasted below:
The device is: GQ GMC-300E plus
SPIR datalength requested: 4095 , hex chars in SPIR command: 0f ff
SPIR data received length: 4095 <type 'str'>
SPIR history data are: (Format: dec byte_number:hex value)
0:55 1:aa 2:00 3:11 4:01 (and so on...)
The device is: GQ GMC-300E plus
SPIR datalength requested: 4096 , hex chars in SPIR command: 10 00
SPIR data received length: 1 <type 'str'>
SPIR history data are: (Format: dec byte_number:hex value)
0:55
The first lines show that a request for 4095 bytes returns 4095 bytes.
The next lines show that a request for 4096 bytes returns only 1 byte (and the serial port runs into a timeout).
If this was meant to return 4096 bytes then I'd say it is a bug?
The test program:
#!/usr/bin/python
# -*- coding: UTF-8 -*-
import serial # the communication with the serial port
def getSPIR(ser, address = 0, datalength = 4096):
ad = chr(0x00) * 3 # for testing address fixed at 0x000000
dlmsb = chr((datalength & 0xff00) >> 8)
dllsb = chr((datalength & 0xff) )
dl = dlmsb + dllsb
print "SPIR datalength requested:\t", datalength, ", hex chars in SPIR command: {:02x} {:02x}".format(ord(dlmsb), ord(dllsb))
ser.write(b'<SPIR'+ad+dl+'>>')
rec = ser.read(datalength)
print "SPIR data received length:\t", len(rec), type(rec)
return rec
def main(argv=None):
print "The device is: \t\t\tGQ GMC-300E plus"
# open the serial port !!! NOTE: baud rate must be set at device first, default is 57600 !!!
ser = serial.Serial('/dev/ttyUSB0', 115200, timeout= 6)
spir = getSPIR(ser, 0 , 4096 - 0)
print "SPIR history data are: \t(Format: dec byte_number:hex value)"
for i in range(0, 16): # limit printout to first 256 bytes
print "\t\t\t\t",
for j in range(0,16):
k = i*16 + j
print "%4d:%02x " % (k, ord(spir[k])),
print
ser.close() # close port
########################################################################
if __name__ == "__main__":
main()
|
|
|
| Reply #4
ullix
Germany
1168 Posts |
Posted - 02/07/2017 : 04:47:50
|
I believe to have figured out what the problem is. Looks like a bug in the firmware.
Upon a SPIR command requesting datalength bytes the device - in my case a "GQ GMC-300E plus" with firmware "GMC-300Re 4.20" - delivers:
[(datalength modulo 4096) + 1]
bytes. So when asking for:
- 0 (zero) bytes, it delivers 1 byte
- 32 bytes, it delivers 33 bytes
- 4096 bytes, it delivers 1 byte (because (4096 modulo 4096) = zero, plus 1 byte = 1 byte; same as first case)
- 4128 bytes, it delivers 33 bytes (because (4128 modulo 4096) + 1 = 32 + 1 = 33 bytes )
Note that 4128 = 4096 + 32 = 0x0fff + 0x0020. The firmware probably ANDs the datalength with 0x000fff and then the bug delivers an extra byte. So, while you can request datalength > 4096, only the moduloed portion will be used.
You can test this yourself easily with my geiger_SPIRtest.py code downloaded from https://sourceforge.net/projects/geigerlog/files/
Remains to be verified with other devices. |
|
|
| Reply #5
ZLM
1261 Posts |
Posted - 02/07/2017 : 19:38:07
|
The firmware limited the length to 4096+1 bytes.
Before read the data, the requested length does a AND binary operation with 0xFFF in hex value.
|
|
|
| Reply #6
ullix
Germany
1168 Posts |
Posted - 02/08/2017 : 07:21:53
|
Actually, the limit is not 4096+1 = 4097 bytes, but 4096. However, you have to ask for 4095 bytes, in order to get 4096!
This bug makes further problems under certain conditions:
E.g., when you ask for 5 bytes from address 0, you send:
<SPIR00005>>
Reading 5 bytes gives you the bytes from addresses 0, 1, 2, 3, and 4. However, the device readies a 6th byte, which you haven't read yet.
Sending another command
<SPIR00005>>
does not result in dumping that extra byte from the first command, but instead this byte is being delivered now with the 2nd read:
Reading again 5 bytes gives you the bytes from addresses 5, 0, 1, 2, 3. Now 2 additional bytes are waiting for being read, and so on.
If this happens within a data sequence, you get spurious readings. If it happens within a tag, your history likely becomes a complete mess.
So, you cannot ignore that extra byte, you MUST make sure that it is read (and handled or discarded). My history data have already suffered from this.
Clearly, this is a bug and needs to be fixed! Will GQ be fixing it?
|
|
|
| Reply #7
wikilicious
USA
22 Posts |
Posted - 11/29/2023 : 19:26:58
|
I'm trying to make a Python package... https://pypi.org/project/pygmc/ and came across this issue.
Requesting 0 bytes and getting 1 byte is what convinced me ullix was right.
I think it's unpythonic to have to deal with this on a device by device basis.
I understand even if I ask for X bytes and only read X bytes... there will be +1 byte waiting in the buffer that will butt in the next read.
Using 'pyserial=3.4' I issue `reset_input_buffer()` & `reset_output_buffer()` and have observed that extra byte go away.
An issue still remains... as ullix pointed out... mod 4096. So I just avoid using max read buffer size.
Resetting both input & output buffers in pyserial had low wall-clock time penalty for a convenient way to deal with this issue. |
|
|
| Reply #8
ullix
Germany
1168 Posts |
Posted - 11/29/2023 : 23:37:03
|
Your package looks interesting! I suggest to also offer a simple logging program along the lines of my simplest GeigerLog varieties, like geigerlog_simple*.py (https://sourceforge.net/projects/geigerlog/files/geigerlog_simple.py/download).
The SPIR thing is annoying, but I believe that in the meantime I have covered all the various exceptions due to firmware modification. See the gdev_gmc_hist.py code of GeigerLog for details.
|
|
|
| Reply #9
Erwin55
Germany
41 Posts |
Posted - 11/30/2023 : 11:04:40
|
| @Ullix, I have the same GMC-300E+ and it works fine, with Win7, Win11, Matlab and Octave. When I'm requesting 4096 bytes I get exact this amount of data. You are also from Germany, so you can send the counter to mine and I'll check it with my systems, if you can do w/o the counter for some days. Only to be sure, that it's really firmware and not something in between the counter and the Geigerlog software. If you are interested in send me a message via your Geigerlog page and I will supply my address. |
|
|
| Reply #10
wikilicious
USA
22 Posts |
|
| Reply #11
ullix
Germany
1168 Posts |
Posted - 12/01/2023 : 03:08:01
|
The younger firmwares do get it right; the problem is to also handle the older ones :-((
|
|
|
| Reply #12
Erwin55
Germany
41 Posts |
Posted - 12/02/2023 : 00:49:14
|
| My counter is also GMC-300Re 4.20 (bought in 2018) and I never had problems downloading a certain block. Hence, I don't understand this issue. Is it the operating system, or are there different firmwares with the same version number? |
|
|
| Reply #13
ullix
Germany
1168 Posts |
Posted - 12/03/2023 : 01:55:29
|
@Erwin55: If you really have the very same counter model, with the very same firmware, this different behavior surely is strange! I'd like to take a closer look.
Could you run GeigerLog started with activated GMC counter using this command:
geigerlog -dvw devel
do a history download and post the resulting files from GeigerLog's data directory "geigerlog.proglog" and "geigerlog.proglog.zip" (if the latter exists).
If you have no other place you can post the files here https://sourceforge.net/p/geigerlog/discussion/easyuploads/
Simply create a new topic.
|
|
|
| Reply #14
Erwin55
Germany
41 Posts |
Posted - 12/04/2023 : 03:03:26
|
| @ullix: I don't have Python and thereafter Geigerlog installed. And to be honest, I'm hesitating to install these programs on my old beast. but my offer, see above, is still valid. And I also have no problems to send you my counter if you prefer this. |
|
|
| Reply #15
ullix
Germany
1168 Posts |
Posted - 12/05/2023 : 06:14:52
|
| GeigerLog with a GMC counter runs even on a Raspberry Pi! |
|
|
| Reply #16
Erwin55
Germany
41 Posts |
Posted - 12/05/2023 : 08:24:37
|
| OK convinced, I will try it the next days and come back to you then. |
|
|
| Reply #17
Erwin55
Germany
41 Posts |
Posted - 12/05/2023 : 09:17:21
|
| @ullix, if it's ok for you, further items regarding Geigerlog on your easyupload page you linked above. |
|
|
| Reply #18
Erwin55
Germany
41 Posts |
|
| Reply #19
ullix
Germany
1168 Posts |
Posted - 12/06/2023 : 00:03:28
|
@Erwin55: Thanks. I looked at your data:
The recording database 'testdataGMC300.hisdb' is perfectly fine.
In the geigerlog.proglog file I found:
GeigerLog has Detected device : 'GMC-300Re 4.20'. Exactly what it was supposed to detect.
These lines:2023-12-05 21:23:49.053 DEBUG : ...117 cfg: Memory : 65536
2023-12-05 21:23:49.059 DEBUG : ...119 cfg: SPIRbugfix : True
2023-12-05 21:23:49.078 DEBUG : ...125 cfg: Variables : CPM1st, CPS1st
2023-12-05 21:23:49.081 DEBUG : ...126 cfg: Serial Port : COM4
tell me that you have manually forced these settings in the configuration file geigerlog.cfg. All of them seem to be correct, as your successful recording proves. However, these are the same values that GeigerLog would have applied had you left the 'auto' setting. As GeigerLog strictly does what the user tells it, you have deprived GeigerLog of testing and perhaps warning the user of issues.
I recommend to always use the 'auto' setting, unless you really require something else!
But in summary, your GMC counter works, it does have the SPIR bug and does require the bugfix, but after GeigerLog applies it, it does work correctly!
|
|
|
| Reply #20
Erwin55
Germany
41 Posts |
Posted - 12/11/2023 : 08:29:33
|
Sorry for my late answer, but I didn't get it managed to apply a RSS to the GMC page and so I waited for some message from your Geigerlog page. The installation of Geigerlog took some time, but was simple. Much simpler than I expected. As I'm using Win7 (mostly) I had to use Python 3.8.9 (hope that's the newest for Win7 support). At the first start of Geigerlog I had some problems, the program didn't want to recognize my counter. Simply unplugging and pluggin again the USB cable solved the problem. After that the counter was recognized, COM and bps corretly applied. The only thing what I changed in geigerlog.cfg was setting GMC = yes.
Maybe I don't understand the problem, but when I request a block of history data from the counter I get exact 4096 valid bytes using Matlab and/or Octave (instrument-control toolbox installed). So help me to understand the problem, I would like to know if there is a bug in the firmware. |
|
|
| Reply #21
ullix
Germany
1168 Posts |
Posted - 12/12/2023 : 01:23:29
|
@Erwin55:
The "GMC-300Re 4.20" does have this firmware bug for SPIR!
Just try to set in the configuration file "GMC_SPIRbugfix = no", and see the failures when you attempt to download history.
If you want to dig even deeper, look into function getGMC_SPIR in file gdev_gmc.py. The comments will explain, and you can use this function in testing software of your own.
|
|
|
| Reply #22
Erwin55
Germany
41 Posts |
Posted - 12/12/2023 : 06:18:47
|
@ullix:
I have downloaded the history of my counter with "GMC_SPIRbugfix = no" and "GMC_SPIRbugfix = yes". Of cause I quitted geigerlog before edilting the cfg-file. I can't see any error. Also the csv saved by GeigerLog shows no strange things. I upload the log- and hisdb-files to the easyupload page. It would be nice if you have a look on these and if possible to point out the failure.
One remark: I edited the cfg-file in the last upload and set the serial port to COM4 manually. Now I set it back to auto and the counter was connected correctly. |
|
|
| Reply #23
ullix
Germany
1168 Posts |
Posted - 12/12/2023 : 23:43:54
|
I'm sure you will have noticed that the two tests had the same outcome, because in both proglog files one finds "GMC_SPIRbugfix:True", i.e. the SPIRbug is fixed in both runs.
This is caused by a GeigerLog bug, which results in always forcing this setting. This bug is likely present for a few years, and had not been noted, because nobody ventured to test the wrong setting anyway. (I told you, GeigerLog knows best!).
So, thanks for discovering it! However, I won't fix this in GL1.4.1, as nobody will need it.
However, it is working correctly, and - I have verified - is allowing you an inappropriate setting in the new 1.5 version of GeigerLog. If you really want to see that you can't use your counter without the SPIRbugfix, then download the latest pre-release of GL1.5, which you find here: https://sourceforge.net/p/geigerlog/discussion/devel/
Currently pre58 is latest. It requires a minimum Python version of 3.8, which I believe you do have.
|
|
|
| Reply #24
Erwin55
Germany
41 Posts |
Posted - 12/13/2023 : 00:48:00
|
| Thanks for this clarification. I'll try it. |
|
|
| Reply #25
Erwin55
Germany
41 Posts |
Posted - 12/13/2023 : 06:56:34
|
| I installed the pre-release of GL1.5 and I set "GMC_SPIRbugfix = no". You are right, a download from the counter was not possible. Hence, I converted the GeigerLog download with "GMC_SPIRbugfix = yes" to a bin-file and compared it with the download of my parser and compared it on bin-level. The GeigerLog output and my parser output are absolute identical. So I close this item for me. |
|
|
| Reply #26
ullix
Germany
1168 Posts |
Posted - 12/13/2023 : 22:14:25
|
| I do recommend to keep the 'auto' setting, like "GMC_SPIRbugfix = auto". |
|
|
| |
Topic |
|
Struggling with the SPIR command
