Silent Bugs in C – Vol. 1: The Therac-25 Case and Hardware Demonstrations

Problem 1: Race Condition

• The software was preemptive multitasking, with several tasks accessing shared global variables.
• There were no adequate synchronization mechanisms (mutexes, semaphores, or atomic operations).
• Flags and control variables were read and written non-atomically, meaning these actions could be interrupted before completion.
• The operator frequently selected mode "X" (X-ray), noticed the mistake, and quickly switched to "E" (Electron).

The bug:
The software took approximately 8 seconds to configure the bending magnets responsible for energy selection. If the operator corrected the mode and pressed Enter within that interval, a race condition occurred between the data-entry task and the hardware-configuration tasks. As a result, the system advanced to the treatment phase before all internal variables had been consistently updated.

Result:

• The interface correctly indicated "Electron" mode, but internally the system configured the machine with the high-energy parameters typical of "X-ray" mode, without the target and scattering filter in position. This resulted in a highly concentrated electron beam of lethal intensity.
• ➡️ Classic silent bug: the system appeared to be in a safe state, while internally it operated under a physically invalid condition.

Problem 2: Variable Overflow (Integer Overflow)

• The software used reduced-size integer variables to store counters, states, and accumulated values related to equipment operation.
• There was no explicit bounds validation before arithmetic operations, implicitly assuming that values would never exceed the capacity of the type.
• The code was written in C, a language that provides no automatic mechanism for detecting integer variable overflow.

The bug:
When an integer variable reaches the maximum value representable by its type and undergoes an additional increment, a variable overflow occurs. In unsigned integers, the value simply "wraps around" back to zero and keeps counting. This behavior happens silently, with no compilation error, no runtime exception, and no warning to the software whatsoever.

Result:

• In the Therac-25, variables related to operational state and treatment parameters could undergo this kind of overflow, allowing the system to advance to dangerous operation phases while appearing to be in a safe state.
• ➡️ Classic silent bug: the internal value of the variable appears valid, but represents a logically impossible state from the standpoint of the system design.

Simplified example in C:

Silent integer variable overflow

    #include <stdint.h>

    int main(void)
    {
        uint8_t counter = 250;

        counter += 10;

        while (counter < 100)
        {
            // unexpected loop
        }

        return 0;
    }
      

What the programmer expects:
The initial value of counter is 250. After incrementing by 10, the expected value would be 260, making the condition of the while false and the loop not execute.

What actually happens:
The counter variable is of type uint8_t, which can only store values between 0 and 255. When attempting to store the value 260, an overflow occurs and the resulting value becomes 4 (260 modulo 256).

• The code compiles normally.
• The program runs without crashing.
• No error is signaled by the system.
• The software's control logic is silently altered.

Result:

• The condition counter < 100 becomes true, causing the system to enter an unexpected loop or follow an unforeseen logic path.

Practical test

The goal of this experiment is to demonstrate, in a practical and observable way on real hardware, the simultaneous occurrence of a race condition and a silent variable overflow in a preemptive multitasking system. The scenario was intentionally simplified to highlight how subtle bugs in C can produce logically impossible states, even though the program remains apparently functional.

Environment and tools used

• AM243x LaunchPad Development Kit (Texas Instruments) development board, based on an ARM Cortex-R5F core, representative of real-time embedded systems with determinism requirements.
• FreeRTOS, running in preemptive mode, enabling real concurrency among multiple tasks.
• Texas Instruments MCU+ SDK, using as a base the UART Echo with FreeRTOS example, employed only as minimal infrastructure for operating system initialization.
• Code Composer Studio (CCS) (version 20.3.0.14__1.9.0), used for compilation, firmware loading, and debugging via JTAG.
• Instrumentation via the CIO (C I/O) mechanism of CCS, in which functions such as DebugP_log are redirected by the debugger over JTAG, allowing internal system behavior to be observed without using a physical UART and without directly interfering with task scheduling.

Central idea of the experiment

This experiment uses a global state variable, shared among multiple tasks in a preemptive multitasking system. This variable represents an internal state of the system and is manipulated concurrently by two independent tasks, without any protection mechanism, such as mutexes, critical sections, or atomic operations.

Each task updates the same global variable using different growth rates and distinct execution periods. One task increments the state slowly and incrementally, while the other applies larger and more frequent increments. This combination intentionally creates a classic race-condition scenario, in which the final value of the variable comes to depend on the non-deterministic order of task execution.

In addition, the state variable was purposely defined with a reduced-width integer type (uint8_t), making the occurrence of silent overflow unavoidable. The observed result is a state that remains valid from the C language's point of view, but that takes on logically impossible values from the system's perspective.

The goal of the experiment is not to trigger explicit failures or crashes, but to demonstrate how silent bugs can arise in seemingly functional systems when global variables are shared unsafely in preemptive environments.

Shared global variable without protection

  volatile uint8_t system_state = 0;
  

The system_state variable represents the global state of the system. The volatile qualifier tells the compiler that the variable's value can change at any moment, preventing it from optimizing access by keeping the value in a register. However, volatile does not guarantee atomicity, does not create memory barriers, and does not replace hardware synchronization mechanisms.

Task A — Incremental state update

  void OperatorTask(void *pvParameters)
  {
      (void)pvParameters;

      for (;;)
      {
          system_state++;   /* Non-atomic write */
          vTaskDelay(pdMS_TO_TICKS(200));
      }
  }
  

This task simulates a gradual interaction with the system, incrementing the global state periodically. The system_state++ operation is not atomic and involves multiple internal steps (read, compute, and write), making it vulnerable to preemption by other tasks.

Task B — Concurrent and more aggressive update

  void ControlTask(void *pvParameters)
  {
      (void)pvParameters;

      for (;;)
      {
          system_state += 10;   /* Concurrent write */
          vTaskDelay(pdMS_TO_TICKS(150));
      }
  }
  

This task represents internal control logic of the system and modifies the same global variable, but with larger increments and higher frequency. Running in parallel with the operator task, it creates a real race condition, in which updates can be overwritten or lost.

Global state monitoring task

  void LoggerTask(void *pvParameters)
  {
      (void)pvParameters;

      for (;;)
      {
          DebugP_log(
              "[tick=%lu] system_state=%u\r\n",
              xTaskGetTickCount(),
              system_state
          );

          vTaskDelay(pdMS_TO_TICKS(500));
      }
  }
  

The monitoring task passively observes the value of the global state and logs its evolution over time via the debugging environment's C I/O (CIO) mechanism. This separation allows evidence of the problem to be collected without directly interfering with the behavior of the tasks that manipulate the state.

Task creation and preemption

  void freertos_main(void *args)
  {
      (void)args;

      xTaskCreate(OperatorTask, "Operator", 1024,
                  NULL, tskIDLE_PRIORITY + 2, NULL);

      xTaskCreate(ControlTask, "Control", 1024,
                  NULL, tskIDLE_PRIORITY + 3, NULL);

      xTaskCreate(LoggerTask, "Logger", 1024,
                  NULL, tskIDLE_PRIORITY + 1, NULL);

      vTaskDelete(NULL);
  }
  

The tasks are created with distinct priorities, ensuring preemption occurs. Since there is no form of synchronization whatsoever, the final value of the global variable comes to depend on the order in which the tasks execute, resulting in race conditions and silent overflows, directly observable in the system logs.