| Author |
 Topic  |
|
|
Searinox
Romania
51 Posts |
 Posted - 10/06/2020 : 06:04:38
|
Firmware version is R2.21.
All dose rate estimation modes appear to be possible to send skyrocketing with if exposed to a high enough dose rate in short enough time. The numbers quickly reach huge values and the device restarts. There is already a thread on this I know.
But...
This issue is made much worse if the device is booted while directly in an environment with a high enough dose rate.
The issue appears to be directly tied to the way the dead time is factored in. The lower the dead time, the higher the threshold required to trigger this. With around 40 microseconds or less dead time, the device eventually settles down to normal when booting into 1.1 mSv/h exposure. Disabling dead time altogether should entirely prevent the issue, at the cost of accuracy at high doses.
In other words the issue is made worse the higher the dead time. My particular device is already set to much lower than the default, at 69 microseconds.
Even with 69 us, booting up while directly facing a dose rate of ~1.1 mSv/h or more renders all estimate modes - from Dynamic to 60 seconds - unable to recover. The numbers quickly skyrocket into tens and hundreds of millisieverts per hour before the device ultimately restarts.
This device is supposed to be able to handle up to around 3-4 mSv/h.
Suppose you are in a high-radiation environment with no way to shield the device, and you try turning it on there. The device will never be able to settle to a consistent number and will restart within seconds, before doing that again, over and over. You do not have enough time to go into the settings and disable the dead time either.
There is already a thread mentioning how this can also happen under normal use, especially with Dynamic estimation, provided that the dose rate is high enough. But being caught off-guard on boot in all modes in a high-radiation environment creates problematic situations where the device may be rendered useless for measuring unless the person can somehow reduce the ambient radiation during the first few seconds of boot.
If the dead time is low enough - but not entirely disabled - the device will initially climb to several mSv/h on boot before slowly descending and ultimately settling on the real value. This leads me to believe that if you could cap how high the added up estimate is allowed to reach and constraining it within a realistic limit of the device itself(say 5-9 mSv/h), it would just do an initial short spike and then settle down without crashing, because once it climbs beyond a certain threshold, it just feedback loops and climbs faster and faster. |
Edited by - Searinox on 10/06/2020 07:41:12
|
|
| Reply #1
Damien68
France
780 Posts |
Posted - 10/06/2020 : 07:55:29
|
this is strange, The dead time compensation is the kind of thing that improves high values measurements as long as it's well adjusted. otherwise it mainly brings delusional inconsistencies accentuated by rapid estimation. to avoid this, the seted deadtime must always be underestimated compared to reality. So you are in right way.
|
Mastery is acquired by studying, with it everything becomes simple |
Edited by - Damien68 on 10/06/2020 08:31:55 |
 |
|
| Reply #2
Searinox
Romania
51 Posts |
Posted - 10/06/2020 : 08:36:22
|
The dead time of 69 us that I am using was determined through repeated measurements of single and then multiple sources of known dose rates, and their expected total. Also several sources indicate a dead time of ~50 us for the SBT-11 tubes, as opposed to the default 144 us of the firmware.
With 144 us the issue occurs at even lower doses. When already running and using short timers or dynamic mode, it sometimes helps to bring the source in slowly as a sudden spike leads to an overestimation that pushes the dose rate over the edge where it climbs without stopping. Even then if exposed to about 1.3 mSv/h in dynamic mode, it will go out of control.
The only solution I have found so far is to both keep the estimate time to 30 or 60 seconds, and to also never boot up the counter in direct exposure to ~700 uSv/h or more. Disabling the dead time altogether fixes the issue in boot, dynamic, and fixed timers. Unfortunately then my known 1.1 mSv/h only reads about ~800 uSv/h and it will be more and more inaccurate as the dose climbs.
Testing the bug should be easy enough, with a firmware of 2.21 make sure dead time is enabled and set to default, and hold the device over a source of ~1 mSv/h as you power it on. Or with a device that is already powered on, set estimation to dynamic and expose it to a strong source.
Even easier: with default settings on 2.21(144 us dead time, 60 seconds estimate) expose the device to something that can produce ~300000 CPM. Doesn't have to be at boot time. Within a few seconds you will notice each screen update showing the dose rate increase faster and faster than the previous update, until it reaches ridiculous numbers. The device then restarts. This is with default settings. 300k CPM is within the range of what this should be capable of handling. |
Edited by - Searinox on 10/06/2020 09:48:52 |
|
|
| Reply #3
Damien68
France
780 Posts |
Posted - 10/06/2020 : 10:47:22
|
OK, is not very clear for SBT11 I see like you official typical deadtime of 50uS @400v but it depend of HV voltage and anode resistor value .
144uS is certanly for 350v this can be coherent but apparently not suitable |
Mastery is acquired by studying, with it everything becomes simple |
Edited by - Damien68 on 10/06/2020 11:11:50 |
|
|
| Reply #4
EmfDev
2250 Posts |
Posted - 10/06/2020 : 12:57:48
|
| Edit: does it only happen in the text idle mode? |
|
|
| Reply #5
ullix
Germany
1171 Posts |
Posted - 10/07/2020 : 01:52:53
|
Who has determined a dead time of 40us or 69us? Are we talking about the dead time of a Geiger counter tube?
See this thread http://www.gqelectronicsllc.com/forum/topic.asp?TOPIC_ID=4598, in partucular Reply#10, where I measured the SBT11A dead time as 250 microssecond.
If you get much shorted ones, show us oscilloscope shots of such a pulse; I don't think this is real.
And what dead time correction are you using? As far as I can see the proper deadtime correction simply cannot go through the roof?
|
|
|
| Reply #6
Damien68
France
780 Posts |
Posted - 10/07/2020 : 07:16:05
|
@ullix, I see your post, 4598, is nice but technically the dead time is the required time after tube decharge to recover the plateau voltage.
electrically the tube is a capacitive charge (equivalent to a capacitor),
source: h**p://embedded-lab.com/blog/making-a-digital-capacitance-meter-using-microcontroller/
So the deadtime is not stricly an characteristique intrinsec of the tube and depend of the tube voltage and of the anode resistor value.
also we can not generalize the measurement of the deadtime made with a GMC-300 which does not necessarily have the same anode resistance.
it is personal but it would have been good to place 4 anode resistors, one for each of 4 anodes of the tube. we could hope to divided by 4 the effect of the deadtime (without changing it), and divide by 4 the decharge current during detection. well that's just a speculation, there may be things that I haven't seen.
|
Mastery is acquired by studying, with it everything becomes simple |
Edited by - Damien68 on 10/08/2020 00:46:39 |
|
|
| Reply #7
Damien68
France
780 Posts |
Posted - 10/07/2020 : 07:32:33
|
@ullix:
the principle of dead time correction is based on the following principle,
the detections are all uncorrelated therefore without any relation between them. so we can follow the following reasoning:
1.we observe the signal for 1 second
2. we count the number of detections N.
now we take into account the dead time
3. we then say that we have not observed the signal for a second but for a time of one second minus the number of detections observed multiplied by the deadtime.
4.We say then that the CPS is not worth N / 1second
but is equal to: N / (1second - N * deadtime)
in any case we will end up dividing by 1 / X with X= 1second - N * deadtime
When N will aproche (1Second/deadtime) X will become small and the error that can be made on X (because of an error on the deadtime) will be reflected 'exponentially' on the result because the derivative of 1 / x is infinite in 0
if deadtime is overestimated this can produce following condition: (N*deadtime >= 1second) and make therefore a division by 0 or obtain a negative number.
sometimes we can also replace the division by a limited development (aX + bX^2 + cX^3 ...) or what, that would be less violent but also undesirable.
now if we force ourselves to underestimate the deadtime, this allows a correction to be made without taking any risk of aberrations.
|
Mastery is acquired by studying, with it everything becomes simple |
Edited by - Damien68 on 10/07/2020 08:21:32 |
|
|
| Reply #8
Searinox
Romania
51 Posts |
Posted - 10/07/2020 : 08:38:53
|
@ullix: A dead time of 250 microseconds would render the device overwhelmed at just a few hundred microsieverts per hour, far less than the device is supposed to cap out at. We could "underestimate" that, but by how much? The default 144 is already an underestimate. The 69 I was using is even less. So is 50. The 40 at which I can no longer get huge estimates is even less. At this point I would need a 4+ millisieverts per hour strong source to stress test it to the limit to see which dead time holds up to maximum tube saturation without taking off to infinity. And then recalibrate the CPM/uSv ratio downwards to remain consistent with my known level check sources.
@Damien68: By that formula I would have needed to reach a few tens of thousands of counts per second in order for the dead time to amplify the real result by that much. I'm around 4000 counts per second tops and at 69 us that would take down the divisor to around 0.7, or a 42% increase in reading. This is far from the dose rates I'm seeing which climb by thousands of times.
EDIT: If upon the next update you are using the -compensated- total CPM to add to the existing CPM instead of adding the -uncompensated- CPM and recalculating the divisor using the new total CPM and time, I can see a positive feedback loop happening.
Let's assume a source with a fixed CPS output X. What will happen is the first reading X will be compensated by a percentage, then the next non-compensated reading which is also X is added to the previous saved compensated CPM and compensated again and the proportion of compensation will now be higher. On the next reading this proportion increases again and you've got a climbing exponential.
Here is an example:
First second, the device displays:
CPS1 / (1 - deadtime * CPS1) = M, then saves that result in memory instead of CPS1
And then the next second the device displays:
(M + CPS2) / (2 - deadtime * (M + CPS2))
M is already amplified by compensation, and will be amplified again on 2nd calculation. This amplification might not be noticeable at lower values but on higher numbers it starts to take off very abruptly.
What should be happening is that the non-compensated CPS should be saved and added, and compensation should be done only on that total, once. So when using the CPS1 from the first second of reading and CPS2.
First second, display:
CPS1 / (1 - deadtime * CPS1)
Next second, display:
(CPS1 + CPS2) / (2 - deadtime * (CPS1 + CPS2))
CPS1 and CPS2 are the non-compensated values.
|
Edited by - Searinox on 10/07/2020 09:25:55 |
|
|
| Reply #9
Damien68
France
780 Posts |
Posted - 10/07/2020 : 10:11:50
|
@Searinox, you are right 42% compensation is not critical.
4000 CPS should start to lower the voltage of the 400v generator and mechanically increase the real deadtime of the tube. the deadtime increases exponentially as the output of the 400v generator approaches plateau voltage of the tube.
there may be compensation mechanisms which come into play and which confuse all this a little. there may be a compensation on the 400v generator control (to boost it), or a compensation on the value of the dead time (in addition to the seted value) taken into account by the algorithms.
the goal is to find the correct settings, often to do this you need to know what's going on. |
Mastery is acquired by studying, with it everything becomes simple |
Edited by - Damien68 on 10/07/2020 10:48:18 |
|
|
| Reply #10
ullix
Germany
1171 Posts |
|
| Reply #11
Damien68
France
780 Posts |
Posted - 10/08/2020 : 01:16:13
|
what i think about all this is that the real tube deadtime is not constant because it depends on the voltage of the 400v generator.
more detection we have, more that will solicit the generator, more its voltage will decrease and more the real deadtime of the tube will lengthen.
if we select a low value for the deadtime:
with high rate detection, after a short time the HV generator will weaken, so the real deadtime of the tube will lengthen. in this conditions, after strating time, we will have a very low correction because the real dead time will become very much bigger that the selected one.
if we select a high value for the deadtime:
at the start, the generator will provide its nominal voltage (400v) we will have a real deadtime shorter than the selected one, so if we have a high level of detections we risk having a largely correction overestimated. but after starting time, when the generator will weaken the correction will be more adequate.
on the GMC500+ (at least on certain versions), to avoid the voltage drop of the generator and therefore the extension of the deadtime in high rate detection condition, there is a correction which is made to boost the generator when there is a high rate of detections. I do not know the algorithm but anyway the correction is surely only partial to be stable/secure and is slightly delayed.
I think this explains what Searinox observes.
at the same time the GMC600 is a very sensitive device, better suited to detect low/medium radiations levels. |
Mastery is acquired by studying, with it everything becomes simple |
Edited by - Damien68 on 10/08/2020 23:34:13 |
|
|
| Reply #12
Searinox
Romania
51 Posts |
Posted - 10/09/2020 : 12:56:23
|
| @ullix: The theoretics of how much dead time is way too low(I myself was initially surprised I had to go so far below the default) doesn't change the reality that if I expose the device to my ~860 uSv/h source with a default time of 144 us, it will quickly climb into the whole sieverts or tens of sieverts and crash. Even with 69 us if I can go to around 1.1 to 1.2 mSv/h the same happens. So what's going on here? |
|
|
| Reply #13
ullix
Germany
1171 Posts |
Posted - 10/10/2020 : 00:04:13
|
All mathematics is pointless when you assume a deadtime shorter than the actual one. Take an oscilloscope and measure. This gives reality.
|
|
|
| Reply #14
Damien68
France
780 Posts |
Posted - 10/10/2020 : 00:20:19
|
| if by crash you mean that the GMC-600 reboots, it is certainly due to a division by 0 or a division result which becomes negative. it is only GQ who can confirm this and add a sanity check in their software. |
Mastery is acquired by studying, with it everything becomes simple |
Edited by - Damien68 on 10/10/2020 00:37:12 |
|
|
| Reply #15
Damien68
France
780 Posts |
Posted - 10/10/2020 : 00:48:57
|
| @ullix, you are absolutely right, but we do not really know what GQ does in the software, it is possible that in their algorithm, in some cases they voluntarily increase the value of the deadtime to compensate other problems. it could also come from elsewhere a software bug or hardware problems (low battery or something else). |
Mastery is acquired by studying, with it everything becomes simple |
Edited by - Damien68 on 10/10/2020 00:55:48 |
|
|
| Reply #16
Damien68
France
780 Posts |
Posted - 10/10/2020 : 00:55:30
|
| we could also connect GeigerLog to the GMC 600 via an audio input, then compare what the GMC600 gives to the results given by Geigerlog via audio port. (normally there is no processing / interpretation on the audio output) |
Mastery is acquired by studying, with it everything becomes simple |
|
|
| Reply #17
Damien68
France
780 Posts |
Posted - 10/10/2020 : 01:26:11
|
It might also be interesting to see the data saved in the flash in CPS mode if it possible.
and see the graph drawn with GeigerLog or with the GQ software after have recovering the data via the USB connection (this time). |
Mastery is acquired by studying, with it everything becomes simple |
|
|
| Reply #18
ullix
Germany
1171 Posts |
Posted - 10/10/2020 : 04:32:15
|
When you use GeigerLog for any purpose, make sure to download the latest version:
I have just released GeigerLog 1.0!
At the usual place: https://sourceforge.net/projects/geigerlog/
You can use the AudioCounter function and look at the audio pulses. Though we don't know whether these audio pulses have the same width, or are narrower, or wider that the Geiger tube pulse.
A helpful tool may also be the newly included stand-alone program GLsoundcheck. It gives a roll-by pulse plot like in an untriggered oscilloscope, like this (for negative pulses, about CPM=400)
|
|
|
| Reply #19
lars.dietrich
Germany
6 Posts |
Posted - 10/13/2020 : 00:04:52
|
Are you sure, that it is related to dead time?
I also observed a full crash and reboot with 600+ and firmware 2.21 and reported it by email. Hope that it will be fixed soon.
It only occurs under very special circumstances:
If one sets the display to „Text“ and select with the S2 key the display option „Current uSv/h: xxxx“ for the second row (problem occurs only for this one option out of the four possibilities!), the system crashes completely and reboots as soon as the reading exceeds 9.999.
Regards,
Lars |
|
|
| Reply #20
Searinox
Romania
51 Posts |
Posted - 10/16/2020 : 06:07:17
|
@lars.dietrich: Yes, indirectly. What actually crashes the counter is reaching ridiculously high values. Those values in turn are only reached when a combination of high CPM and dead time happens. The higher the dead time, the lower the CPM at which it goes nuts. Disabling it altogether makes it impossible to do this. I've put sources with a combined ~2 mSv/h on the tube and although it capped out before reaching that high, it no longer crashed and considering I am close to saturating the tube, I am sure.
Do you mean 9.999 uSv/h or mSv/h? If it's mSv/h then yes we could be looking at the same issue since my counter goes way past that number. Also weird because I don't think either variant of the GMC-600 is supposed to be able to read that high. I did also notice that once it goes in the millisievert range, the screen updates become slower and slower. If there's some single-threaded stuff going on with the counting, then another explanation might be that the lag is causing it to perceive shorter and shorter time between the seconds, thus artificially assuming there is a lot more radiation coming in than there is, which produces greater numbers, which adds more lag, and so on. |
Edited by - Searinox on 10/16/2020 06:11:30 |
|
|
| Reply #21
Damien68
France
780 Posts |
Posted - 10/16/2020 : 07:53:43
|
@Searinox, you are probably find origine of the issue and have right, if the screen updates become slower and slower, it can delay the "1 seconde" time base in ratios of 1.5 or more, make primary CPS over evaluated in the same ratio and make exponential error after dead time corection.
certanly the pulses must be counted by generating an interrupt.
The interrupts take up CPU resource and therefore with high count rate the CPU (with the remaining resources) must take more than one second to perform the job that it must do every second. thats delay LCD refresh and in same time certainly extends the "1 seconde" time base, induces an over evaluated count like you said, and make a delusional result after dead time corection.
maybe by disabling wifi or other functions it will save CPU time.
But CPU can be clocked up to 72MHz via a internal PLL so it's strange it should have plenty of time. but maybe the clock is reduced for power saving.
I would not be surprised if in the Hit-IT (detection triggered) it do not just increment an count register but that it also do something else as recalibration of the HV pwm control that can take time.
It would also explain that the device reboot when it has no more CPU time to execute his main task and to refresh his watchdog. |
Mastery is acquired by studying, with it everything becomes simple |
Edited by - Damien68 on 10/16/2020 10:43:10 |
|
|
| Reply #22
Searinox
Romania
51 Posts |
Posted - 10/16/2020 : 11:15:02
|
| @Damien68: I already have wifi disabled. I don't use any other special functionality. Logging is also turned off. |
Edited by - Searinox on 10/16/2020 19:40:28 |
|
|
| Reply #23
Damien68
France
780 Posts |
Posted - 10/16/2020 : 22:36:20
|
| I think it's just a bug in the firmware architecture or coding. certainy in relation with recalibration of the HV pwm control. maybe they are waiting for a synchronization with the pwm output to safety adjust its duty-cycle instead of using an interrupt. or something like this. or something else. |
Mastery is acquired by studying, with it everything becomes simple |
Edited by - Damien68 on 10/17/2020 02:33:38 |
|
|
| Reply #24
ullix
Germany
1171 Posts |
Posted - 10/17/2020 : 00:33:46
|
@Searinox, @lars.dietrich: I cannot quite follow: are you two experienceing the same problem, or only something similar?
What concerns me is that you both seem to refer to a limit 9.999 in a Sievert unit.
Such a magic number would almost always point to a software issue. Therfore, are you seeing the same behaviour when the display is in CPM or CPS?
|
|
|
| Reply #25
Searinox
Romania
51 Posts |
Posted - 10/17/2020 : 07:07:25
|
| @ullix: I am using "large font" mode with display in uSv/h. What I'm referring to is the issue that my device goes into very high values then restarts. What lars is referring to is the device restarting after going over a certain number "9.999" though they did not specify if that's uSv/h or mSv/h. My issue automatically implies his issue since the device goes to numbers way past both 9.999 uSv/h or 9.999 mSv/h. I've not had any issues going over 9.999 uSv/h though if my device goes over 9.999 mSv/h then it's already gone crazy and is bound to restart. Damien pointed out that in fact these could be a whole different issue: going to high numbers in the mSv/h range slows down the display refresh more and more. At more than 1 mSv/h I can tell that it starts to take more than 1 second and then longer and longer for the screen to refresh the dose rate. And he has noted that if this delay becomes too big, a watchdog task on the counter designed to make sure the firmware doesn't become unresponsive catches this condition and restarts the device. |
|
|
| Reply #26
Damien68
France
780 Posts |
Posted - 10/17/2020 : 08:39:29
|
@Searinox: I also think that if the CPU spends too much time in interuption, and thus can't carry out any more its main task in the allotted time this will artificially increase the primary CPS (CPS without dead time corection).
main task must be trigged each seconds by a semaphore or other things and so it calculate CPS by simply taking counts of the number of pulses received between each execution time (each seconds).
if the main task execution time becomes too long (> 1 second) the main task will no longer be executed every second but every 1.5 or 2 seconds. also the primary CPS count will do is no longer count of pulses during 1 seconde but is count of pulses during 1.5 or 2 seconds. but for the CPU it will still be CPS (count during 1 second).
which will give an error of only x1.5 or x2 but which will become strongly exponential after correcting the dead time.
it's just speculation but it corresponds quite well with what we observe. But what is sure at 95% is if the screen refresh is slowed down, the primary CPS ends up overevaluated in the same proportions because it is no longer the count of pulses during one second, and is no longer compatible to be adjusted with its deadtime. |
Mastery is acquired by studying, with it everything becomes simple |
Edited by - Damien68 on 10/18/2020 00:50:12 |
|
|
| Reply #27
ullix
Germany
1171 Posts |
Posted - 10/18/2020 : 01:49:10
|
@Serinox: the other part of may question was whether this overflow is only seen when displaying Sievert values, or also when displaying counts (CPM; CPS)?
Determining the true count rate e.g. via audio counts might still be helpful. |
|
|
| Reply #28
Damien68
France
780 Posts |
Posted - 10/18/2020 : 06:18:57
|
@ullix, sure, using GeigerLog via audio port we should be able to obtain a decent result. |
Mastery is acquired by studying, with it everything becomes simple |
Edited by - Damien68 on 10/18/2020 06:24:32 |
|
|
| Reply #29
lars.dietrich
Germany
6 Posts |
Posted - 10/20/2020 : 01:25:48
|
So my issue is definitely a firmware issue. It happens, if I exceed a reading of 9.999 uS/h in text mode and with "Current uSv/h: xxxx" displayed in the second row. If I display anything else there or switch to any other display like "Large", it does not crash.
I tried to get nearer and nearer to a source, until it crashes. It is always when the reading goes above 9.999 uSv/h. It may run for hours on levels of 9.90 to 9.999 uSv/h, but in the same second where the level is exceeded to 10 uSV/h it crashes. In e.g. "Large" mode I can go way beyond the limit without any issues.
|
|
|
| Reply #30
Searinox
Romania
51 Posts |
Posted - 11/01/2020 : 00:27:11
|
| It only just clicked to me that the thread TOPIC_ID=9130 and this are basically reporting the same issue. I think the devs need a strong source(around 1mSv/h or more) to stress test the device themselves. Since you're US-based, I can recommend a Cs-137 source from United Nuclear or a WW2 radium luminous disk. |
Edited by - Searinox on 11/01/2020 00:33:19 |
|
|
| Reply #31
Searinox
Romania
51 Posts |
Posted - 11/14/2020 : 03:33:29
|
@lars.dietrich: I can also confirm your bug. If the unit is set in Text Mode and the Current uSv/h reading exceeds 9.999 uSv/h, the entire screen stops updating and the unit freezes up, then restarts after a few seconds. That one seems to be unrelated to the unit being overwhelmed in its calculations by too much radiation and is just a straight-up bug with the mode.
Devs please keep us posted on firmware updates. :D |
Edited by - Searinox on 11/14/2020 03:37:02 |
|
|
| Reply #32
EmfDev
2250 Posts |
Posted - 11/16/2020 : 10:37:44
|
| We have tested this and can confirm the issue, we are now finding a way to fix this. |
|
|
| |
Topic |
|
GMC 600 R 2.21 crash on ALL Estimation Modes
