ChipSHOUTER with ChipWhisperer Lite external trigger

[esanfelix]

Hi everyone

I’m trying to setup some experiments with the ChipSHOUTER with an external hardware trigger provided by the CW-Lite. The main issue is I’m getting ‘fault_trigger_glitch’ almost immediately and I’m not sure why this is happening. I also get quite a few temperature faults.

I’ve seen threads on very similar topics, but couldn’t find a solution there so hopefully someone can help me out here Note I’m quite new to ChipWhisperer and ChipSHOUTER, so I may have very obvious configuration problems.

Let’s start by describing the setup and seeing if the configuration makes sense.

I got an Arduino target running some basic counter loop. The Arduino triggers the CW-Lite through a GPIO connected to TIO4, and generates a glitch based off that external trigger. CW-Lite also controls the reset pin of the Arduino via TIO3. I’ve verified this all works fine by monitoring the TIO4 trigger and the HS2 signal (after setting it to ‘glitch’).

The ChipWhisperer-Lite Glitch connector is connected to the trigger input on the ChipSHOUTER.

I start by doing the following on the scope side:

        # Connect to cw lite scope
        if self._scope is None:
            self._scope = cw.scope()
        
        # This is required to interact with the ChipShouter
        self._scope.glitch.output = "enable_only"
        self._scope.io.glitch_lp = True
        
        self._scope.glitch.repeat = 1
        self._scope.glitch.trigger_src = "ext_single"

        # We trigger from "tio4" input
        self._scope.trigger.triggers = "tio4"

        # Arm glitch before scope module
        self._scope.glitch.arm_timing = "before_scope"
        
        # CLKGEN frequency as timebase for glitches
        self._scope.glitch.clk_src = 'clkgen'

        # Ensure our reset signal does not start asserted
        self._scope.io.tio3 = "gpio_high"

        # Route glitch signal to hs2 for monitoring
        self._scope.io.hs2 = "glitch"

Next I initialize the ChipSHOUTER:

        # Disarm if we need to
        if self._shouter.armed:
            self._shouter.armed = False

        # Setup external trigger mode for the chipwhisperer
        self._shouter.hwtrig_term = False
        self._shouter.hwtrig_mode = False

And after this I setup my parameters (these change for each attempt), arm the scope and the shouter before sending a command to my target:

            # Setup trigger parameters  
            self._scope.glitch.ext_offset =  off
            self._scope.glitch.repeat = width

            # Setup shouter voltage
            self._shouter.voltage = volt

           # Arm scope and then shouter
           self._scope.arm()
           # Clear errors and arm shouter
           self._shouter.faults_current = 0
           self._shouter.armed = True

After my attempt is done I disarm the scope and the shouter.

As you can see I’m even clearing the errors before arming, yet it’s still failing to arm because of a latched error.

If I disconnect the external trigger from the ChipSHOUTER, I can run everything smoothly without errors, but I’m obviously not injecting any faults … this makes me think the configuration of the glitch output from CW-Lite is incorrect but I cannot figure out what exactly is wrong.

Any pointers on what may be wrong with the setup are appreciated!

[jpthibault]

How long is the CS trigger pulse? This error will happen if the trigger pulse exceeds 10ms (see ChipSHOUTER® Python API Documentation — ChipSHOUTER 1.0.0 documentation).

[esanfelix]

Hi,

Thanks for your answer @jpthibault .

I have to run more experiments later, and I’m away from the setup ATM. However in my understanding what you’re referring to is ‘fault_trigger_error’ while I’m seeing ‘fault _trigger_glitch’.

According to the docs that means:

faults_trig_glitch - Trigger was attempted while device disarmed

But I don’t really understand what is causing that in my setup.

[esanfelix]

So I came back now to look into it. I can run with these settings for 100 iterations without triggering the issue:

[+] Setting glitch parameters
cwlite Device
sn         = 53313120313436373230352037323038
fw_version = 
    major = 0
    minor = 62
    debug = 0
gain = 
    mode = low
    gain = 0
    db   = 5.5
adc = 
    state          = False
    basic_mode     = low
    timeout        = 2
    offset         = 0
    presamples     = 0
    samples        = 24400
    decimate       = 1
    trig_count     = 1818921813
    fifo_fill_mode = normal
clock = 
    adc_src       = clkgen_x1
    adc_phase     = 0
    adc_freq      = 96000000
    adc_rate      = 96000000.0
    adc_locked    = True
    freq_ctr      = 0
    freq_ctr_src  = extclk
    clkgen_src    = system
    extclk_freq   = 10000000
    clkgen_mul    = 2
    clkgen_div    = 1
    clkgen_freq   = 192000000.0
    clkgen_locked = True
trigger = 
    triggers = tio4
    module   = basic
io = 
    tio1         = serial_tx
    tio2         = serial_rx
    tio3         = gpio_high
    tio4         = high_z
    pdid         = high_z
    pdic         = high_z
    nrst         = high_z
    glitch_hp    = False
    glitch_lp    = True
    extclk_src   = hs1
    hs2          = glitch
    target_pwr   = True
    tio_states   = (1, 0, 1, 0)
    cdc_settings = bytearray(b'\x01\x00\x00\x00')
glitch = 
    clk_src     = clkgen
    width       = 10.15625
    width_fine  = 0
    offset      = 10.15625
    offset_fine = 0
    trigger_src = ext_single
    arm_timing  = before_scope
    ext_offset  = 10
    repeat      = 1
    output      = enable_only

[+] Setting up shouter voltage to 312
api_version         = 0.0.0
armed               = False
voltage = 
    set      = 312
    measured = 21
pulse = 
    width        = 160
    repeat       = 3
    deadtime     = 10
    actual width = 160
state               = disarmed
trigger_safe        = False
faults_current      = []
faults_latched      = []
temperature_mosfet  = 31
temperature_diode   = 30
temperature_xformer = 35
arm_timeout         = 2
hwtrig_term         = False
hwtrig_mode         = False
emode               = False
mute                = True
absent_temp         = 1
pat_enable          = False
pat_wave            = 0111000000111110
reset_config        = 0
reset               = False
board_id            = P1DESG:2.0.2

boot_enable         = 0

The HS2 signal then looks as follows:

If I now double the scope.glitch.repeat value and set it to 2, it crashes almost immediately.

[+] Setting glitch parameters
cwlite Device
sn         = 53313120313436373230352037323038
fw_version = 
    major = 0
    minor = 62
    debug = 0
gain = 
    mode = low
    gain = 0
    db   = 5.5
adc = 
    state          = False
    basic_mode     = low
    timeout        = 2
    offset         = 0
    presamples     = 0
    samples        = 24400
    decimate       = 1
    trig_count     = 1948065733
    fifo_fill_mode = normal
clock = 
    adc_src       = clkgen_x1
    adc_phase     = 0
    adc_freq      = 96000000
    adc_rate      = 96000000.0
    adc_locked    = True
    freq_ctr      = 0
    freq_ctr_src  = extclk
    clkgen_src    = system
    extclk_freq   = 10000000
    clkgen_mul    = 2
    clkgen_div    = 1
    clkgen_freq   = 192000000.0
    clkgen_locked = True
trigger = 
    triggers = tio4
    module   = basic
io = 
    tio1         = serial_tx
    tio2         = serial_rx
    tio3         = gpio_high
    tio4         = high_z
    pdid         = high_z
    pdic         = high_z
    nrst         = high_z
    glitch_hp    = False
    glitch_lp    = True
    extclk_src   = hs1
    hs2          = glitch
    target_pwr   = True
    tio_states   = (1, 0, 1, 0)
    cdc_settings = bytearray(b'\x01\x00\x00\x00')
glitch = 
    clk_src     = clkgen
    width       = 10.15625
    width_fine  = 0
    offset      = 10.15625
    offset_fine = 0
    trigger_src = ext_single
    arm_timing  = before_scope
    ext_offset  = 10
    repeat      = 2
    output      = enable_only

[+] Setting up shouter voltage to 420
api_version         = 0.0.0
armed               = False
voltage = 
    set      = 420
    measured = 21
pulse = 
    width        = 160
    repeat       = 3
    deadtime     = 10
    actual width = 160
state               = fault_disarmed
trigger_safe        = False
faults_current      = ['fault_trigger_glitch']
faults_latched      = []
temperature_mosfet  = 31
temperature_diode   = 31
temperature_xformer = 35
arm_timeout         = 2
hwtrig_term         = False
hwtrig_mode         = False
emode               = False
mute                = True
absent_temp         = 1
pat_enable          = False
pat_wave            = 0111000000111110
reset_config        = 0
reset               = False
board_id            = P1DESG:2.0.2

boot_enable         = 0

[+] Arming scope
faults latched:  []
faults current:  ['fault_trigger_glitch']
faults latched after clear:  []
faults current after clear:  ['fault_trigger_glitch']
[+] Arming shouter
Traceback (most recent call last):
[...]
  File "/home/user/.pyenv/versions/cw/lib/python3.9/site-packages/chipshouter/chipshouter.py", line 112, in __setattr__
    super(DisableNewAttr, self).__setattr__(name, value)
  File "/home/user/.pyenv/versions/cw/lib/python3.9/site-packages/chipshouter/chipshouter.py", line 617, in armed
    self.com_api.cmd_arm()
  File "/home/user/.pyenv/versions/cw/lib/python3.9/site-packages/chipshouter/com_tools.py", line 1122, in cmd_arm
    raise Firmware_State_Exception("State:" + self.get_state() + str(self.get_faults_latched()))
chipshouter.com_tools.Firmware_State_Exception: State:fault_disarmed[]

The HS2 pulse looks like this on my scope now:

This is also far from 10ms… but it looks like anything over 1 will cause the error to occur.

Just to be sure, do I understand correctly that the error I’m seeing occurs when a hardware trigger is detected while the ChipSHOUTER is disarmed?

I’ll probably need to spend some time troubleshooting and looking at all the signals. Do you perhaps have a script performing a similar task that’s known to work and you could share?

Thanks,
Eloi

[Hello_Friend]

@esanfelix

The ChipWhisperer-Lite Glitch connector is connected to the trigger input on the ChipSHOUTER.

Might be a silly thing to check, but the glitch connector is tied to the mosfet(s), so what you see on HS2 may be different to what the chipshouter sees. You may need to connect HS2 to the trigger input instead?

[esanfelix]

Uhm I was reading the “External trigger and ChipWhisperer Glitch Output” on the manual that came with my chipshouter (Oct 2021 version).

I was using HS2 as a reference of how the Glitch Output will be driven even if it’s not what the chipshouter sees.

But you’re right that I could try to connect the HS2 output and use it as an active-high trigger instead. I had skimmed through the other manual section and jumped to the “how do I use it with CW” part as I understood that was recommended.

The manual does say that the recommended way is to connect HS2 of the Advanced Breakout Board, which I don’t have and is currently out of stock. I’ll try driving it directly, but first I need to get a few cables and connectors as I don’t really have a proper lab.

Anyway, thanks for the help. I’ll report back once I tried or if I have any further ideas / progress.

[esanfelix]

By the way, I’ve made a standalone script that requires no target to trigger the issue I’m having, in case you’d like to try to reproduce.

The code is as follows:

import chipwhisperer as cw
from chipshouter import ChipSHOUTER
import sys
import time

port = sys.argv[1] if len(sys.argv) > 1 else "/dev/ttyUSB0"
width = sys.argv[2] if len(sys.argv) > 2 else 1
off = sys.argv[3] if len(sys.argv) > 3 else 1
volt = 250


print("[+] Connecting to chipshouter on {:1}".format(port))
shouter = ChipSHOUTER(port)
print("[+] Connecting to scope")
scope = cw.scope()


#Initialize scope

print("[+] Setting up scope")

scope.glitch.output = "enable_only"
scope.io.glitch_lp = True
scope.glitch.repeat = 1
scope.trigger.triggers = "tio4"
scope.glitch.clk_src = 'clkgen'
scope.io.tio3 = "gpio_high"
scope.io.hs2 = "glitch"
scope.glitch.arm_timing = "before_scope"
scope.glitch.trigger_src = "manual" # XXX ext_single in real use case

# Initialize shouter
if shouter.armed:
    shouter.armed = False

shouter.hwtrig_term = False
shouter.hwtrig_mode = False

for i in range(100):
    # Configure trigger

    print("[+] Running test {:1}".format(i))

    scope.glitch.ext_offset =  off
    scope.glitch.repeat = width

    # Configure shouter
    if shouter.armed:
        shouter.armed = False

    # Setup and arm
    print("[+] Setting up shouter voltage to " + str(int(volt)))
    shouter.voltage = volt

    # Arm scope
    scope.glitch.output = "enable_only"
    scope.io.glitch_lp = True
    print(scope)

    # scope.arm() # XXX Only when using external trigger

    # Arm shouter
    print("faults latched: ", shouter.faults_latched)
    print("faults current: ", shouter.faults_current)
    shouter.faults_current = 0
    print("faults latched after clear: ", shouter.faults_latched)
    print("faults current after clear: ", shouter.faults_current)
    print("[+] Arming shouter")
    shouter.armed = True

    # In real tests we do a scope.arm() here and ensuring it gets an external trigger
    # by interacting with the target later
    # Here we just sleep half a second and do a manual trigger
    time.sleep(0.5)
    scope.glitch.manual_trigger()

    # And now disarm stuff and repeat
    shouter.armed = False

Calling this like python test.py /dev/ttyUSB0 1 runs the 100 iterations without triggering the error. Running with python test.py /dev/ttyUSB0 2 fails within 1-2 iterations.

The physical setup is as described earlier, but since the target is not used the only relevant connection is between the SMA “Glitch” connector from the ChipWhisperer and the SMB “Trig” connector of the ChipSHOUTER.

[esanfelix]

Alright, I’m making some progress and continuing my monologue in case it’s useful to someone else.

I borrowed an SMB cable with stripped ends from someone nearby and attached directly to HS2 as mentioned before. I also changed the trigger mode to active high, as the HS2 signal is active high. This didn’t immediately fix the problem.

I then added a wait for shouter.trigger_safe after the loop, and now I’m actually glitching the Arduino like this from a standalone script without that error. I am however reading temperature sensor errors every now and then.

This script does not seem to trigger the problem:

import chipwhisperer as cw
from chipshouter import ChipSHOUTER
import sys
import time

port = sys.argv[1] if len(sys.argv) > 1 else "/dev/ttyUSB0"
width = sys.argv[2] if len(sys.argv) > 2 else 1
off = sys.argv[3] if len(sys.argv) > 3 else 1
volt = 250


print("[+] Connecting to chipshouter on {:1}".format(port))
shouter = ChipSHOUTER(port)
print("[+] Connecting to scope")
scope = cw.scope()


#Initialize scope

print("[+] Setting up scope")

scope.glitch.output = "enable_only"
scope.io.glitch_lp = True
scope.glitch.repeat = 1
scope.trigger.triggers = "tio4"
scope.glitch.clk_src = 'clkgen'
scope.io.tio3 = "gpio_high"
scope.io.hs2 = "glitch"
scope.glitch.arm_timing = "before_scope"
scope.glitch.trigger_src = "manual" # XXX ext_single in real use case

shouter.mute = True

# Initialize shouter
if shouter.armed:
    shouter.armed = False

shouter.hwtrig_term = False # High impedance
shouter.hwtrig_mode = True # Active high

for i in range(100):
    # Configure trigger

    print("[+] Running test {:1}".format(i))

    scope.glitch.ext_offset =  off
    scope.glitch.repeat = width

    # Configure shouter
    if shouter.armed:
        shouter.armed = False

    # Setup and arm
    print("[+] Setting up shouter voltage to " + str(int(volt)))
    shouter.voltage = volt

    # scope.arm() # XXX Only when using external trigger

    # Arm shouter
    print("faults latched: ", shouter.faults_latched)
    print("faults current: ", shouter.faults_current)

    print("[+] Arming shouter")
    shouter.armed = True
    # Wait for trigger safe?
    while not shouter.trigger_safe:
        print("\t waiting for trigger_safe")
        time.sleep(0.4)

    # In real tests we do a scope.arm() here and ensuring it gets an external trigger
    # by interacting with the target later
    # Here we just sleep half a second and do a manual trigger
    # input("Enter!")
    print("[+] Triggering!")
    scope.glitch.manual_trigger()

    # And now disarm stuff and repeat
    shouter.armed = False

I’ve also modified this and added my target code, setting the CW-Lite up such that it triggers based off the input trigger coming from the target. I do immediately start seeing glitches and getting wrong values out (I’m just doing a simple loop test) as well as large data dumps. So it seems to work well enough though sometimes results in fault_temp_sensor.

Questions:

1. Is this the intended use of shouter.trigger_safe ?

2. It takes quite a while each time it’s arm for trigger_safe to be true (like close to a second). Would it make sense to arm before the loop and leave it armed until the experiment is done? Is it safe to change the voltage parameter of the chipshouter while it’s armed? It feels like it should be better (safer) to disarm and rearm just before glitching, but adding 1 second to each iteration of my loop does not sound like a good thing either for more complex targets.

3. When running the experiments with my target attached, I sometimes get fault_temp_sensor after glitching. Anything I can do to prevent this?

[jpthibault]

Thanks for the updates. The intended use of trigger_safe is explained here, but your code is fine. ChipSHOUTER® Python API Documentation — ChipSHOUTER 1.0.0 documentation

Your code works for me with the CW glitch output. What you could try is to increase the shouter.absent_temp setting; this may address your fault_temp_sensor problem.

You can get much faster glitching by removing the arming/de-arming from the main loop. You’ll probably want a small delay between glitches to ensure CS remains in an operable state. And of course for maximum glitch performance you’ll need forced-air cooling. But even without cooling, I can do 1000’s of glitches at many glitches/second (how much will of course depend on all the glitch parameters).

Jean-Pierre

[esanfelix]

Thanks Jean-Pierre.

Thanks for the updates. The intended use of trigger_safe is explained here, but your code is fine. ChipSHOUTER® Python API Documentation — ChipSHOUTER 1.0.0 documentation

Yeah I did read that line in the documentation, but it’s not clear to me if this is supposed to be used in the way I’m using it (i.e. waiting for it to be True before actually triggering the device).

Your code works for me with the CW glitch output. What you could try is to increase the shouter.absent_temp setting; this may address your fault_temp_sensor problem.

Just to be sure, are you referring to the version with the trigger_safe wait, or the other one? I’ll take a look at increasing shouter.absent_temp.

You can get much faster glitching by removing the arming/de-arming from the main loop. You’ll probably want a small delay between glitches to ensure CS remains in an operable state. And of course for maximum glitch performance you’ll need forced-air cooling. But even without cooling, I can do 1000’s of glitches at many glitches/second (how much will of course depend on all the glitch parameters).

Yeah I tried that and it did indeed improve the throughput, but resulted in the CS resetting every 30-50 attempts or so. Before I stopped yesterday I saw on the documentation that this is somewhat expected so I’ll just catch the exception and give the CS some time to reboot when it happens.

Thanks again!

[jpthibault]

@esanfelix

Yeah I tried that and it did indeed improve the throughput, but resulted in the CS resetting every 30-50 attempts or so. Before I stopped yesterday I saw on the documentation that this is somewhat expected so I’ll just catch the exception and give the CS some time to reboot when it happens.

If you increase the delay between glitches sufficiently, you should be able to reach the point where you can run glitching for a very long time without issues. With your script I was able to glitch thousands of times with 0.2s between glitches.

[esanfelix]

With or without the trigger_safe wait?

[jpthibault]

With the trigger_safe wait loop, but it was never entered (i.e. it always returned True). Calling trigger_safe is recommended, even if it always returns True right away.

[esanfelix]

That’s interesting because for me it usually returns False for 3-4 iterations at least. I wonder where that difference comes from…

[coflynn]

Hi @esanfelix - sorry on the lag, several of us were travelling for Embedded World and with current travel issues we’ve been substantially delayed! But one quick note:

@Alex_Dewar had fixed one issue we were seeing where the temp sensor was locking up. What firmware are you on?

The latest are available at https://github.com/newaetech/ChipSHOUTER/blob/master/documentation/Firmware%20Update.md (the FW files are unique per unit, so just use the URL format described in there to get your units FW update).