Reduce memory usage of debug_trace* calls

[ahamlat]

While evaluating performance of debug_trace* calls in Besu, we observed significant overhead caused by memory copying in the trace frames, even when opcodes do not mutate EVM memory.

A representative scenario illustrates the issue:

opcode 1     (stack only)
opcode 2     (stack only)
opcode 3     (stack only)
opcode 4     (touches memory: KECCAK256)
opcode 5     (stack only)
...
opcode 20000 (stack only)

In the example above:

Only opcode 4 performs a memory expansion, and subsequent opcodes do not interact with memory.

However, in Besu’s current trace implementation, each trace frame copies the entire memory segment even when the memory has not changed since the previous frame. This results in:

• Repeated full-memory copies
• High allocation churn
• Increased GC pressure
• Severe slowdowns in large traces
• Significant overhead when debug tracing high-gas transactions, especially when the opcode touching memory appears in the beginning of the transaction

Essentially, the same memory snapshot is redundantly cloned thousands of times.

Expected Behavior

When tracing :

• Trace frames should reference the previous memory snapshot unless the opcode modifies memory.
• Only opcodes that explicitly write to memory (e.g., MSTORE, MSTORE8, SHA3/KECCAK256 preimage build) should trigger a memory snapshot update.

[ahamlat]

We need to avoid the case from the screenshot below, the opcode corresponding to index 15 is KECCAK256, and from that Opcode, all trace frames consume 495424 bytes, where 495200 is memory (the same copie again and again).