加密器与检测方对抗的具体实验,附 C 实现与分类器代码
恶意软件密码学机器学习
﷽
Hello, cybersecurity enthusiasts and white hackers!
In
part 42
, I implemented Speck to transform a payload before restoring it. In
part 37
and
part 38
, we looked at nonlinearity, output differences, and S-box properties. Today I want to connect these ideas to a small machine-learning experiment:
can a tiny neural network recognize a statistical relationship between two encrypted blocks?
We will implement a reference cipher in C, build a simple statistical classifier, and train a small neural network on ciphertext pairs. The GPU experiment is intended for my NVIDIA L20. An optional final example asks a local Qwen model to explain the measurements.
The target here is
Speck32/64 with a configurable number of rounds
. Part 42 used
Speck128/128
. These are different members of the same family: changing the word size also changes the rotation constants, key schedule parameters, and full round count.
This PoC post includes the runnable experiment. NVIDIA L20 accuracy and timing results were obtained from an actual run on my VM.
the idea
Suppose we encrypt one block and then encrypt another block with a small, fixed difference in its input. After only a few rounds, that relationship may still leave a measurable pattern in the outputs.
Let the first plaintext be \(P\), and choose a fixed XOR difference \(\Delta\):
\[P' = P \oplus \Delta\]
Encrypt both blocks using the same key within that pair:
\[C = E_K^{(r)}(P), \qquad C' = E_K^{(r)}(P')\]
Here, \(r\) is the number of rounds. The superscript means “run the first \(r\) rounds of this cipher,” not exponentiation.
We create two equally sized classes:
class 1:
the plaintexts have the selected difference;
class 0:
the plaintexts are independent uniform random blocks, encrypted under the same key within the pair.
The model receives only the ciphertexts. It never receives the plaintexts, the key, or the label as an input feature. Each pair has a newly generated key.
For a fixed key, encryption is a permutation of the block space. Encrypting two independent uniform inputs therefore produces two independent uniform outputs. This makes the second class a useful control, while keeping both classes on the same encryption path.
Our input difference, written as two 16-bit words, is:
Delta
=
(
0x0040, 0x0000
)
This experiment is inspired by Aron Gohr’s
research on neural distinguishers for round-reduced Speck32/64
. His work studies much more sophisticated models and attacks. Here we build a smaller classifier and measure its limits; the results of that paper are not measurements of our implementation.
For malware research, this is useful when reviewing a custom decoder or a modified cryptographic routine: a random-looking byte distribution does not tell us whether relationships between inputs survived the transformation. The experiment assumes that we can generate the chosen plaintext pairs. It is not a method for identifying arbitrary encrypted blobs from their bytes alone.
Speck32/64 in a few equations
Speck32/64 uses two 16-bit data words, a 64-bit key, and
22
rounds in its full configuration. Its rotation constants are
7
and
2
. The original cipher description and test vectors are available through the
designers’ publications page
.
One round transforms the words \((x,y)\) using a round key \(k_i\):
\[x' = \left(\operatorname{ROR}_{16}(x,7) + y \pmod{2^{16}}\right) \oplus k_i\]
\[y' = \operatorname{ROL}_{16}(y,2) \oplus x'\]
Addition, rotation, XOR: this is an
ARX
construction. The modular addition introduces carry interactions between bits. Those interactions are one reason that examining each output bit separately can miss useful structure.
The inverse round is also short:
\[y = \operatorname{ROR}_{16}(y' \oplus x',2)\]
\[x = \operatorname{ROL}_{16}\left((x' \oplus k_i)-y \pmod{2^{16}},7\right)\]
We will check the implementation before generating any training data. A classifier can learn artifacts from an incorrect cipher just as happily as it learns patterns from the intended one.
practical example 1. reference cipher in C
First, the rotations. Intermediate calculations use
uint32_t
; converting back to
uint16_t
keeps the low 16 bits:
static
uint16_t
rol16
(
uint16_t
x
,
unsigned
n
)
{
return
(
uint16_t
)(((
uint32_t
)
x
<<
n
)
|
((
uint32_t
)
x
>>
(
16
-
n
)));
}
static
uint16_t
ror16
(
uint16_t
x
,
unsigned
n
)
{
return
(
uint16_t
)(((
uint32_t
)
x
>>
n
)
|
((
uint32_t
)
x
<<
(
16
-
n
)));
}
These helpers are only called with the fixed, nonzero rotation amounts used by the cipher.
Next, the key schedule. We interpret the least significant 16-bit key word as the first round key. For the example key
1918111009080100
, the internal words are
0100
,
0908
,
1110
, and
1918
:
rk
[
0
]
=
(
uint16_t
)
key
;
l
[
0
]
=
(
uint16_t
)(
key
>>
16
);
l
[
1
]
=
(
uint16_t
)(
key
>>
32
);
l
[
2
]
=
(
uint16_t
)(
key
>>
48
);
for
(
unsigned
i
=
0
;
i
<
FULL_ROUNDS
-
1
;
++
i
)
{
l
[
i
+
3
]
=
(
uint16_t
)((
uint16_t
)(
ror16
(
l
[
i
],
7
)
+
rk
[
i
])
^
i
);
rk
[
i
+
1
]
=
(
uint16_t
)(
rol16
(
rk
[
i
],
2
)
^
l
[
i
+
3
]);
}
Keeping the word order explicit avoids a common problem: implementing the correct arithmetic with the key words reversed.
The encryption loop follows the round equations directly:
for
(
unsigned
i
=
0
;
i
<
rounds
;
++
i
)
{
x
=
(
uint16_t
)((
uint16_t
)(
ror16
(
x
,
7
)
+
y
)
^
rk
[
i
]);
y
=
(
uint16_t
)(
rol16
(
y
,
2
)
^
x
);
}
To decrypt, walk through the round keys backwards, undo the second word first, and then undo the first word. The command-line interface limits the round count to
1..22
.
The known-answer check is:
key: 1918111009080100
plaintext: 6574694c
ciphertext: a86842f2
rounds: 22
The block is displayed as
left_word || right_word
. These are hexadecimal integer words, so no host-dependent byte casting is involved in the CLI. The same vector is included in
Gohr’s reference implementation
.
Now the complete full source code of our PoC (
hack.c
):
/*
* hack.c - Speck32/64 reference for the neural distinguisher lab.
* author: @cocomelonc
* words are printed left first; key[0] is the least significant word.
*/
#include
<errno.h>
#include
<inttypes.h>
#include
<stdint.h>
#include
<stdio.h>
#include
<stdlib.h>
#include
<string.h>
#define FULL_ROUNDS 22
static
uint16_t
rol16
(
uint16_t
x
,
unsigned
n
)
{
return
(
uint16_t
)(((
uint32_t
)
x
<<
n
)
|
((
uint32_t
)
x
>>
(
16
-
n
)));
}
static
uint16_t
ror16
(
uint16_t
x
,
unsigned
n
)
{
return
(
uint16_t
)(((
uint32_t
)
x
>>
n
)
|
((
uint32_t
)
x
<<
(
16
-
n
)));
}
static
void
expand_key
(
uint64_t
key
,
uint16_t
rk
[
FULL_ROUNDS
])
{
uint16_t
l
[
FULL_ROUNDS
+
2
];
rk
[
0
]
=
(
uint16_t
)
key
;
l
[
0
]
=
(
uint16_t
)(
key
>>
16
);
l
[
1
]
=
(
uint16_t
)(
key
>>
32
);
l
[
2
]
=
(
uint16_t
)(
key
>>
48
);
for
(
unsigned
i
=
0
;
i
<
FULL_ROUNDS
-
1
;
++
i
)
{
l
[
i
+
3
]
=
(
uint16_t
)((
uint16_t
)(
ror16
(
l
[
i
],
7
)
+
rk
[
i
])
^
i
);
rk
[
i
+
1
]
=
(
uint16_t
)(
rol16
(
rk
[
i
],
2
)
^
l
[
i
+
3
]);
}
static
uint32_t
encrypt
(
uint32_t
p
,
const
uint16_t
*
rk
,
unsigned
rounds
)
{
uint16_t
x
=
(
uint16_t
)(
p
>>
16
),
y
=
(
uint16_t
)
p
;
for
(
unsigned
i
=
0
;
i
<
rounds
;
++
i
)
{
x
=
(
uint16_t
)((
uint16_t
)(
ror16
(
x
,
7
)
+
y
)
^
rk
[
i
]);
y
=
(
uint16_t
)(
rol16
(
y
,
2
)
^
x
);
}
return
((
uint32_t
)
x
<<
16
)
|
y
;
}
static
uint32_t
decrypt
(
uint32_t
c
,
const
uint16_t
*
rk
,
unsigned
rounds
)
{
uint16_t
x
=
(
uint16_t
)(
c
>>
16
),
y
=
(
uint16_t
)
c
;
for
(
unsigned
i
=
rounds
;
i
>
0
;
--
i
)
{
y
=
ror16
((
uint16_t
)(
y
^
x
),
2
);
x
=
rol16
((
uint16_t
)((
uint16_t
)(
x
^
rk
[
i
-
1
])
-
y
),
7
);
}
return
((
uint32_t
)
x
<<
16
)
|
y
;
}
static
int
read_number
(
const
char
*
s
,
int
base
,
uint64_t
max
,
uint64_t
*
out
)
{
char
*
end
;
if
(
!
s
[
0
]
||
s
[
0
]
==
'-'
||
s
[
0
]
==
'+'
||
s
[
0
]
==
' '
)
return
0
;
errno
=
0
;
uintmax_t
value
=
strtoumax
(
s
,
&
end
,
base
);
if
(
errno
||
end
==
s
||
*
end
||
value
>
max
)
return
0
;
*
out
=
(
uint64_t
)
value
;
return
1
;
}
int
main
(
int
argc
,
char
**
argv
)
{
uint16_t
rk
[
FULL_ROUNDS
];
if
(
argc
==
1
||
(
argc
==
2
&&
strcmp
(
argv
[
1
],
"selftest"
)
==
0
))
{
expand_key
(
UINT64_C
(
0x1918111009080100
),
rk
);
uint32_t
c
=
encrypt
(
UINT32_C
(
0x6574694c
),
rk
,
FULL_ROUNDS
);
if
(
c
!=
UINT32_C
(
0xa86842f2
)
||
decrypt
(
c
,
rk
,
FULL_ROUNDS
)
!=
UINT32_C
(
0x6574694c
))
{
fputs
(
"[=^..^=] Speck32/64 vector FAILED
\n
"
,
stderr
);
return
1
;
}
printf
(
"[=^..^=] Speck32/64: 6574694c -> %08"
PRIx32
" -> 6574694c
\n
"
,
c
);
return
0
;
}
uint64_t
rounds
,
key
,
block
;
if
(
argc
!=
5
||
(
strcmp
(
argv
[
1
],
"enc"
)
&&
strcmp
(
argv
[
1
],
"dec"
))
||
!
read_number
(
argv
[
2
],
10
,
FULL_ROUNDS
,
&
rounds
)
||
rounds
==
0
||
!
read_number
(
argv
[
3
],
16
,
UINT64_MAX
,
&
key
)
||
!
read_number
(
argv
[
4
],
16
,
UINT32_MAX
,
&
block
))
{
fprintf
(
stderr
,
"usage: %s [selftest | enc|dec rounds key_hex block_hex]
\n
"
,
argv
[
0
]);
return
2
;
}
expand_key
(
key
,
rk
);
uint32_t
result
=
strcmp
(
argv
[
1
],
"enc"
)
==
0
?
encrypt
((
uint32_t
)
block
,
rk
,
(
unsigned
)
rounds
)
:
decrypt
((
uint32_t
)
block
,
rk
,
(
unsigned
)
rounds
);
printf
(
"%08"
PRIx32
"
\n
"
,
result
);
return
0
;
}
demo 1
Let’s look this in practice. Build:
gcc
-O2
-std
=
c11
-Wall
-Wextra
-Wpedantic
hack.c
-o
hack
then run:
./hack selftest
./hack enc 22 1918111009080100 6574694c
./hack dec 22 1918111009080100 a86842f2
As you can see, the last two commands must print
a86842f2
and
6574694c
, respectively. These values are fixed by the test vector, rather than performance measurements.
For a reduced-round example:
./hack enc 3 1918111009080100 6574694c
Use its printed ciphertext with
./hack dec 3 ...
to reverse the same three rounds. A successful round trip is useful, but the published vector is the stronger check: matching bugs in encryption and decryption can otherwise cancel each other.
practical example 2. from output differences to a tiny neural network
We need three pieces: a data generator, a statistical baseline, and a small model. Each will be explained before the full script.
step 1 - vectorize the cipher
In PyTorch, a word is an
int64
tensor whose values stay in
0..65535
. Using a wider integer type makes the mask after addition explicit:
x
=
((
ror
(
x
,
7
)
+
y
)
&
0xffff
)
^
k
y
=
rol
(
y
,
2
)
^
x
Each tensor position belongs to an independent sample. PyTorch executes these operations over the batch on the selected device. We use integer operations for the cipher; floating-point arithmetic only appears when the resulting bits enter the classifier.
The script checks the published vector on the chosen device. With
--reference ./hack
, it also compares C and PyTorch encryption for deterministic random keys and inputs at several round counts, and checks the C decryption result.
step 2 - construct the two classes
The positive pair starts with the selected XOR difference. The negative pair uses the independently generated second block:
delta
=
torch
.
tensor
(
DIFF
,
device
=
device
).
reshape
(
2
,
1
)
q
=
torch
.
where
(
labels
.
bool
().
unsqueeze
(
0
),
p
^
delta
,
independent
)
a
,
b
=
encrypt
(
p
[
0
],
p
[
1
],
key
,
rounds
)
c
,
d
=
encrypt
(
q
[
0
],
q
[
1
],
key
,
rounds
)
The ciphertext words are \((a,b,c,d)\). Notice that both blocks of a pair use the same key for both classes.
We generate keys for the whole experiment together, check that every 64-bit key pattern is unique, and then divide them into training, validation, and test sets. This gives a concrete separation:
no key in the test split appears in training or validation
.
The simulation uses seeded pseudorandom tensors so that the experiment can be repeated. This is a dataset generator, not a key-generation API for deployed cryptographic software. Each split has equal numbers of positive and negative samples.
step 3 - turn ciphertext words into bits
Feeding a 16-bit word as one floating-point number would impose an artificial numerical distance: values
32767
and
32768
are close as integers but differ in every bit. Instead, expose the individual bits:
def
bits
(
words
):
shifts
=
torch
.
arange
(
15
,
-
1
,
-
1
,
device
=
words
.
device
)
return
((
words
.
T
.
unsqueeze
(
-
1
)
>>
shifts
)
&
1
).
flatten
(
1
).
float
()
The default network input is
64
bits: two ciphertexts, each consisting of two 16-bit words. We also keep the
32
bits of their XOR difference:
\[D = C \oplus C'\]
The command
--features xor
gives only these difference bits to the neural model. This second experiment will help us separate the effect of nonlinear learning from the effect of giving the model more information.
step 4 - build a baseline we can understand
Before training a network, estimate how often each output-difference bit is set in each class. With a smoothing count of one:
\[p_{yj} = \frac{1 + \sum_{i:Y_i=y}D_{ij}}{2 + N_y}\]
Here, \(N_y\) is the number of training examples in class \(y\). Smoothing prevents taking the logarithm of zero when a bit never changes in a small dataset.
Treat the bits as independent Bernoulli variables and form a log-likelihood ratio:
\[s(D)=\sum_j\left[D_j\log\frac{p_{1j}}{p_{0j}}+(1-D_j)\log\frac{1-p_{1j}}{1-p_{0j}}\right]\]
The classes have equal prior probability, so predict class
1
when \(s(D)\geq 0\). For example, if a bit has probabilities
0.9
and
0.5
in the two classes, seeing a
1
adds about
0.588
to the score. Seeing a
0
contributes about
-1.609
. Those numbers illustrate the equation; they are not measured Speck probabilities.
This score can be written as a dot product plus a bias:
w
=
torch
.
log
(
p1
/
p0
)
-
torch
.
log
((
1
-
p1
)
/
(
1
-
p0
))
b
=
torch
.
log
((
1
-
p1
)
/
(
1
-
p0
)).
sum
()
scores
=
difference
@
w
+
b
This is an independent-bit baseline. It does not enumerate a full differential distribution table, and it ignores dependencies between bits. Its purpose is to tell us whether the neural model adds anything beyond simple bit biases.
step 5 - add a small nonlinear model
The model has two hidden layers:
64 input bits -> 128 ReLU -> 64 ReLU -> 1 logit
In equations:
\(h_1=\operatorname{ReLU}(W_1u+b_1)\)
\(h_2=\operatorname{ReLU}(W_2h_1+b_2)\)
\(z=W_3h_2+b_3,\qquad p=\frac{1}{1+e^{-z}}\)
The default version has
16,641
trainable parameters. With XOR-only inputs, it has
12,545
. This is an ordinary multilayer perceptron, or MLP:
class
TinyNet
(
nn
.
Module
):
def
__init__
(
self
,
width
):
super
().
__init__
()
self
.
layers
=
nn
.
Sequential
(
nn
.
Linear
(
width
,
128
),
nn
.
ReLU
(),
nn
.
Linear
(
128
,
64
),
nn
.
ReLU
(),
nn
.
Linear
(
64
,
1
))
def
forward
(
self
,
x
):
return
self
.
layers
(
x
).
squeeze
(
1
)
We minimize binary cross-entropy:
\[L=-\frac{1}{N}\sum_i\left[y_i\log p_i+(1-y_i)\log(1-p_i)\right]\]
For a positive sample, a prediction of
0.9
contributes about
0.105
to the loss; a prediction of
0.2
contributes about
1.609
. Gradient descent updates the weights to reduce these mistakes.
The implementation uses
BCEWithLogitsLoss
, which combines the sigmoid and cross-entropy calculation in a numerically stable form. Therefore, the network returns the raw logit
z
; we do not put an extra sigmoid inside the model.
PyTorch documentation
.
step 6 - choose the model without looking at the test answers
After each epoch, compute validation loss. Keep the weights from the epoch with the lowest validation loss. Only then evaluate those weights on the test split. The baseline is fitted using training data alone.
The script records accuracy, true-positive rate, false-positive rate, and an approximate
95%
Wilson interval for accuracy. Since both test classes have the same size, ordinary accuracy equals balanced accuracy. The interval describes sampling uncertainty for one fixed model; it does not describe variation between training runs.
The full training script (
train.py
):
#!/usr/bin/env python3
"""Speck32/64 neural distinguisher lab. author: @cocomelonc."""
import
argparse
import
csv
import
hashlib
import
json
import
math
from
pathlib
import
Path
import
platform
import
random
import
subprocess
import
sys
import
time
import
torch
from
torch
import
nn
MASK
=
0xffff
DIFF
=
(
0x0040
,
0x0000
)
def
rol
(
x
,
n
):
return
((
x
<<
n
)
|
(
x
>>
(
16
-
n
)))
&
MASK
def
ror
(
x
,
n
):
return
((
x
>>
n
)
|
(
x
<<
(
16
-
n
)))
&
MASK
def
encrypt
(
x
,
y
,
key
,
rounds
):
# key[0] is the low word; the other three words form the l sequence.
k
=
key
[
0
]
l
=
[
key
[
1
],
key
[
2
],
key
[
3
]]
for
i
in
range
(
rounds
):
x
=
((
ror
(
x
,
7
)
+
y
)
&
MASK
)
^
k
y
=
rol
(
y
,
2
)
^
x
if
i
+
1
<
rounds
:
new_l
=
((
ror
(
l
[
i
],
7
)
+
k
)
&
MASK
)
^
i
l
.
append
(
new_l
)
k
=
rol
(
k
,
2
)
^
new_l
return
x
,
y
def
selftest
(
device
,
reference
=
None
):
key
=
torch
.
tensor
([[
0x0100
],
[
0x0908
],
[
0x1110
],
[
0x1918
]],
device
=
device
)
x
,
y
=
encrypt
(
torch
.
tensor
([
0x6574
],
device
=
device
),
torch
.
tensor
([
0x694c
],
device
=
device
),
key
,
22
)
if
(
x
.
item
(),
y
.
item
())
!=
(
0xa868
,
0x42f2
):
raise
RuntimeError
(
"Speck32/64 published vector failed"
)
if
reference
:
executable
=
str
(
Path
(
reference
).
resolve
(
strict
=
True
))
rng
=
random
.
Random
(
20261028
)
for
rounds
in
(
1
,
2
,
3
,
5
,
6
,
22
):
for
_
in
range
(
4
):
k
,
p
=
rng
.
getrandbits
(
64
),
rng
.
getrandbits
(
32
)
words
=
torch
.
tensor
([[(
k
>>
(
16
*
i
))
&
MASK
]
for
i
in
range
(
4
)],
device
=
device
)
a
,
b
=
encrypt
(
torch
.
tensor
([
p
>>
16
],
device
=
device
),
torch
.
tensor
([
p
&
MASK
],
device
=
device
),
words
,
rounds
)
expected
=
(
a
.
item
()
<<
16
)
|
b
.
item
()
got
=
subprocess
.
check_output
(
[
executable
,
"enc"
,
str
(
rounds
),
f
"
{
k
:
016
x
}
"
,
f
"
{
p
:
08
x
}
"
],
text
=
True
)
plain
=
subprocess
.
check_output
(
[
executable
,
"dec"
,
str
(
rounds
),
f
"
{
k
:
016
x
}
"
,
got
.
strip
()],
text
=
True
)
if
int
(
got
,
16
)
!=
expected
or
int
(
plain
,
16
)
!=
p
:
raise
RuntimeError
(
f
"C / PyTorch disagreement at
{
rounds
}
rounds"
)
print
(
f
"[=^..^=] cipher checks passed on
{
device
}
"
,
flush
=
True
)
def
raw_splits
(
sizes
,
seed
):
# Generate a fixed corpus on CPU, independent of the selected GPU.
g
=
torch
.
Generator
().
manual_seed
(
seed
)
total
=
sum
(
sizes
)
while
True
:
keys
=
torch
.
randint
(
0
,
65536
,
(
4
,
total
),
generator
=
g
)
# A signed int64 still uniquely represents each 64-bit pattern.
packed
=
keys
[
0
]
|
(
keys
[
1
]
<<
16
)
|
(
keys
[
2
]
<<
32
)
|
(
keys
[
3
]
<<
48
)
if
torch
.
unique
(
packed
).
numel
()
==
total
:
break
p
=
torch
.
randint
(
0
,
65536
,
(
2
,
total
),
generator
=
g
)
independent
=
torch
.
randint
(
0
,
65536
,
(
2
,
total
),
generator
=
g
)
splits
,
start
=
[],
0
for
size
in
sizes
:
labels
=
torch
.
cat
((
torch
.
zeros
(
size
//
2
),
torch
.
ones
(
size
//
2
)))
labels
=
labels
[
torch
.
randperm
(
size
,
generator
=
g
)]
sl
=
slice
(
start
,
start
+
size
)
splits
.
append
((
keys
[:,
sl
],
p
[:,
sl
],
independent
[:,
sl
],
labels
))
start
+=
size
return
splits
def
bits
(
words
):
shifts
=
torch
.
arange
(
15
,
-
1
,
-
1
,
device
=
words
.
device
)
return
((
words
.
T
.
unsqueeze
(
-
1
)
>>
shifts
)
&
1
).
flatten
(
1
).
float
()
def
make_data
(
raw
,
rounds
,
device
):
key
,
p
,
independent
,
labels
=
(
v
.
to
(
device
)
for
v
in
raw
)
delta
=
torch
.
tensor
(
DIFF
,
device
=
device
).
reshape
(
2
,
1
)
q
=
torch
.
where
(
labels
.
bool
().
unsqueeze
(
0
),
p
^
delta
,
independent
)
a
,
b
=
encrypt
(
p
[
0
],
p
[
1
],
key
,
rounds
)
c
,
d
=
encrypt
(
q
[
0
],
q
[
1
],
key
,
rounds
)
pair
=
bits
(
torch
.
stack
((
a
,
b
,
c
,
d
)))
difference
=
bits
(
torch
.
stack
((
a
^
c
,
b
^
d
)))
return
pair
,
difference
,
labels
def
fit_bit_baseline
(
difference
,
labels
):
# smoothed Bernoulli likelihoods for each output-difference bit.
p
=
[]
for
value
in
(
0
,
1
):
subset
=
difference
[
labels
==
value
]
p
.
append
((
subset
.
sum
(
0
)
+
1
)
/
(
subset
.
shape
[
0
]
+
2
))
p0
,
p1
=
p
w
=
torch
.
log
(
p1
/
p0
)
-
torch
.
log
((
1
-
p1
)
/
(
1
-
p0
))
b
=
torch
.
log
((
1
-
p1
)
/
(
1
-
p0
)).
sum
()
return
w
,
b
class
TinyNet
(
nn
.
Module
):
def
__init__
(
self
,
width
):
super
().
__init__
()
self
.
layers
=
nn
.
Sequential
(
nn
.
Linear
(
width
,
128
),
nn
.
ReLU
(),
nn
.
Linear
(
128
,
64
),
nn
.
ReLU
(),
nn
.
Linear
(
64
,
1
))
def
forward
(
self
,
x
):
return
self
.
layers
(
x
).
squeeze
(
1
)
@
torch
.
no_grad
()
def
predict
(
model
,
data
,
batch
):
model
.
eval
()
return
torch
.
cat
([
model
(
x
)
for
x
in
data
.
split
(
batch
)])
def
measure
(
logits
,
labels
):
predicted
=
logits
>=
0
truth
=
labels
.
bool
()
n
=
labels
.
numel
()
acc
=
(
predicted
==
truth
).
float
().
mean
().
item
()
# approximate 95% Wilson interval for the fraction of correct decisions.
z
=
1.959963984540054
den
=
1
+
z
*
z
/
n
center
=
(
acc
+
z
*
z
/
(
2
*
n
))
/
den
half
=
z
*
math
.
sqrt
(
acc
*
(
1
-
acc
)
/
n
+
z
*
z
/
(
4
*
n
*
n
))
/
den
return
dict
(
accuracy
=
acc
,
ci_low
=
center
-
half
,
ci_high
=
center
+
half
,
tpr
=
predicted
[
truth
].
float
().
mean
().
item
(),
fpr
=
predicted
[
~
truth
].
float
().
mean
().
item
())
def
synchronize
(
device
):
if
device
.
type
==
"cuda"
:
torch
.
cuda
.
synchronize
(
device
)
def
run_round
(
args
,
raw
,
rounds
,
device
):
synchronize
(
device
)
started
=
time
.
perf_counter
()
train
,
valid
,
test
=
[
make_data
(
r
,
rounds
,
device
)
for
r
in
raw
]
synchronize
(
device
)
generation_seconds
=
time
.
perf_counter
()
-
started
torch
.
manual_seed
(
args
.
seed
+
rounds
)
if
args
.
shuffle_labels
:
# Destroy the feature/label relationship separately in EVERY split.
train
,
valid
,
test
=
[
(
pair
,
diff
,
labels
[
torch
.
randperm
(
labels
.
numel
(),
device
=
device
)])
for
pair
,
diff
,
labels
in
(
train
,
valid
,
test
)
]
index
=
0
if
args
.
features
==
"pair"
else
1
x
,
y
=
train
[
index
],
train
[
2
]
w
,
b
=
fit_bit_baseline
(
train
[
1
],
y
)
baseline
=
measure
(
test
[
1
]
@
w
+
b
,
test
[
2
])
model
=
TinyNet
(
x
.
shape
[
1
]).
to
(
device
)
optimizer
=
torch
.
optim
.
Adam
(
model
.
parameters
(),
lr
=
args
.
lr
)
loss_fn
=
nn
.
BCEWithLogitsLoss
()
best_loss
,
best_state
,
best_epoch
,
best_acc
=
math
.
inf
,
None
,
0
,
0.0
synchronize
(
device
)
started
=
time
.
perf_counter
()
for
epoch
in
range
(
1
,
args
.
epochs
+
1
):
model
.
train
()
order
=
torch
.
randperm
(
y
.
numel
(),
device
=
device
)
total_loss
=
torch
.
zeros
((),
device
=
device
)
for
ids
in
order
.
split
(
args
.
batch_size
):
optimizer
.
zero_grad
(
set_to_none
=
True
)
loss
=
loss_fn
(
model
(
x
[
ids
]),
y
[
ids
])
loss
.
backward
()
optimizer
.
step
()
total_loss
+=
loss
.
detach
()
*
ids
.
numel
()
scores
=
predict
(
model
,
valid
[
index
],
args
.
batch_size
)
val_loss
=
loss_fn
(
scores
,
valid
[
2
]).
item
()
val_acc
=
measure
(
scores
,
valid
[
2
])[
"accuracy"
]
if
val_loss
<
best_loss
:
best_loss
,
best_epoch
,
best_acc
=
val_loss
,
epoch
,
val_acc
best_state
=
{
k
:
v
.
detach
().
cpu
().
clone
()
for
k
,
v
in
model
.
state_dict
().
items
()}
print
(
f
"rounds=
{
rounds
:
2
d
}
epoch=
{
epoch
:
2
d
}
loss=
{
total_loss
.
item
()
/
y
.
numel
()
:
.
5
f
}
"
f
"val_loss=
{
val_loss
:
.
5
f
}
val_acc=
{
val_acc
:
.
4
f
}
"
,
flush
=
True
)
synchronize
(
device
)
training_seconds
=
time
.
perf_counter
()
-
started
model
.
load_state_dict
(
best_state
)
neural
=
measure
(
predict
(
model
,
test
[
index
],
args
.
batch_size
),
test
[
2
])
checkpoint
=
dict
(
state_dict
=
best_state
,
rounds
=
rounds
,
features
=
args
.
features
,
input_width
=
x
.
shape
[
1
],
difference
=
list
(
DIFF
),
seed
=
args
.
seed
,
best_epoch
=
best_epoch
,
shuffle_labels
=
args
.
shuffle_labels
,
baseline_w
=
w
.
detach
().
cpu
(),
baseline_b
=
b
.
detach
().
cpu
())
torch
.
save
(
checkpoint
,
args
.
out
/
f
"rounds-
{
rounds
}
.pt"
)
common
=
dict
(
rounds
=
rounds
,
seed
=
args
.
seed
,
n_test
=
test
[
2
].
numel
(),
shuffled_labels
=
args
.
shuffle_labels
,
generation_seconds
=
generation_seconds
)
rows
=
[
dict
(
common
,
method
=
"bit_baseline"
,
features
=
"xor"
,
**
baseline
,
validation_accuracy
=
""
,
best_epoch
=
""
,
parameters
=
33
,
training_seconds
=
""
),
dict
(
common
,
method
=
"tiny_mlp"
,
features
=
args
.
features
,
**
neural
,
validation_accuracy
=
best_acc
,
best_epoch
=
best_epoch
,
parameters
=
sum
(
p
.
numel
()
for
p
in
model
.
parameters
()),
training_seconds
=
training_seconds
)]
print
(
f
"[=^..^=] rounds=
{
rounds
}
: baseline=
{
baseline
[
'accuracy'
]
:
.
4
f
}
; "
f
"MLP=
{
neural
[
'accuracy'
]
:
.
4
f
}
; test keys=
{
test
[
2
].
numel
()
}
"
,
flush
=
True
)
return
rows
def
main
():
parser
=
argparse
.
ArgumentParser
(
description
=
__doc__
)
parser
.
add_argument
(
"--device"
,
choices
=
(
"cpu"
,
"cuda"
),
default
=
"cuda"
)
parser
.
add_argument
(
"--self-test"
,
action
=
"store_true"
)
parser
.
add_argument
(
"--reference"
,
help
=
"path to compiled hack.c"
)
parser
.
add_argument
(
"--rounds"
,
nargs
=
"+"
,
type
=
int
,
default
=
[
2
,
3
,
4
,
5
,
6
,
22
])
parser
.
add_argument
(
"--features"
,
choices
=
(
"pair"
,
"xor"
),
default
=
"pair"
)
parser
.
add_argument
(
"--shuffle-labels"
,
action
=
"store_true"
)
parser
.
add_argument
(
"--train-size"
,
type
=
int
,
default
=
1048576
)
parser
.
add_argument
(
"--val-size"
,
type
=
int
,
default
=
65536
)
parser
.
add_argument
(
"--test-size"
,
type
=
int
,
default
=
262144
)
parser
.
add_argument
(
"--epochs"
,
type
=
int
,
default
=
15
)
parser
.
add_argument
(
"--batch-size"
,
type
=
int
,
default
=
8192
)
parser
.
add_argument
(
"--lr"
,
type
=
float
,
default
=
0.001
)
parser
.
add_argument
(
"--seed"
,
type
=
int
,
default
=
42
)
parser
.
add_argument
(
"--cpu-threads"
,
type
=
int
,
default
=
4
)
parser
.
add_argument
(
"--out"
,
type
=
Path
,
default
=
Path
(
"results"
))
args
=
parser
.
parse_args
()
sizes
=
[
args
.
train_size
,
args
.
val_size
,
args
.
test_size
]
if
any
(
n
<
128
or
n
%
2
for
n
in
sizes
):
parser
.
error
(
"split sizes must be even and at least 128"
)
if
any
(
r
<
1
or
r
>
22
for
r
in
args
.
rounds
)
or
len
(
set
(
args
.
rounds
))
!=
len
(
args
.
rounds
):
parser
.
error
(
"round counts must be distinct integers in 1..22"
)
if
min
(
args
.
epochs
,
args
.
batch_size
,
args
.
cpu_threads
)
<
1
or
not
0
<
args
.
lr
<
1
:
parser
.
error
(
"epochs, batch size and threads must be positive; 0 < lr < 1"
)
if
not
0
<=
args
.
seed
<
2
**
31
:
parser
.
error
(
"seed must be in 0..2^31-1"
)
if
args
.
device
==
"cuda"
and
not
torch
.
cuda
.
is_available
():
parser
.
error
(
"CUDA unavailable: install a CUDA PyTorch wheel, or explicitly use --device cpu"
)
torch
.
set_num_threads
(
args
.
cpu_threads
)
device
=
torch
.
device
(
args
.
device
)
selftest
(
device
,
args
.
reference
)
if
args
.
self_test
:
return
args
.
out
.
mkdir
(
parents
=
True
,
exist_ok
=
False
)
metadata
=
dict
(
arguments
=
{
k
:
str
(
v
)
if
isinstance
(
v
,
Path
)
else
v
for
k
,
v
in
vars
(
args
).
items
()},
python
=
platform
.
python_version
(),
torch
=
torch
.
__version__
,
cuda
=
torch
.
version
.
cuda
,
platform
=
platform
.
platform
(),
device
=
torch
.
cuda
.
get_device_name
(
0
)
if
device
.
type
==
"cuda"
else
"CPU"
,
difference
=
list
(
DIFF
),
status
=
"running"
,
command
=
sys
.
argv
,
source_sha256
=
hashlib
.
sha256
(
Path
(
__file__
).
read_bytes
()).
hexdigest
())
metadata_path
=
args
.
out
/
"metadata.json"
metadata_path
.
write_text
(
json
.
dumps
(
metadata
,
indent
=
2
)
+
"
\n
"
)
print
(
f
"[=^..^=] device:
{
metadata
[
'device'
]
}
; preparing disjoint keys"
,
flush
=
True
)
raw
=
raw_splits
(
sizes
,
args
.
seed
)
with
(
args
.
out
/
"metrics.csv"
).
open
(
"w"
,
newline
=
""
)
as
stream
:
writer
=
None
for
rounds
in
args
.
rounds
:
rows
=
run_round
(
args
,
raw
,
rounds
,
device
)
if
writer
is
None
:
writer
=
csv
.
DictWriter
(
stream
,
fieldnames
=
list
(
rows
[
0
]))
writer
.
writeheader
()
writer
.
writerows
(
rows
)
stream
.
flush
()
metadata
[
"status"
]
=
"complete"
metadata_path
.
write_text
(
json
.
dumps
(
metadata
,
indent
=
2
)
+
"
\n
"
)
print
(
f
"[=^..^=] saved results to
{
args
.
out
}
"
,
flush
=
True
)
if
__name__
==
"__main__"
:
main
()
demo 2. running on the L20
Let’s check this in my NVIDIA host. The source directory includes
hack.c
,
train.py
,
plot.py
,
explain.py
, a
Makefile
. Run the following commands from that directory on the GPU machine.
First, inspect the environment:
nvidia-smi
create virtual env:
python3
-m
venv .venv
.
.venv/bin/activate
python
-m
pip
install
--upgrade
pip
Install a CUDA-enabled PyTorch build compatible with the machine’s driver and Python. For example, the official version table lists this CUDA 12.6 build:
python
-m
pip
install
torch
==
2.13.0
--index-url
https://download.pytorch.org/whl/cu126
python
-m
pip
install
-r
requirements.txt
My server has only just been set up, which is why it took some time and a large amount of data had to be downloaded:
Use the
PyTorch installation selector
or
version table
if your environment requires another wheel. These scripts use standard tensor operations and do not require a separate
nvcc
build.
On a machine with several GPUs, select an available card. Here,
0
is an example index; replace it with the index or UUID of the L20 you intend to use:
CUDA_VISIBLE_DEVICES
=
0 python
-c
'import torch; print(torch.__version__, torch.version.cuda, torch.cuda.is_available()); print(torch.cuda.get_device_name(0))'
CUDA availability in my VM:
make
CUDA_VISIBLE_DEVICES
=
0 python train.py
--device
cuda
--self-test
--reference
./hack
python
-m
pip freeze
>
environment.txt
nvidia-smi
-q
>
environment-gpu.txt
So, as you can see,
nvidia-smi
showing the selected L20, CUDA availability, and successful C/PyTorch agreement checks.
Start with a short functional run:
CUDA_VISIBLE_DEVICES
=
0 python train.py
--device
cuda
--rounds
2 22
\
--train-size
8192
--val-size
1024
--test-size
2048
\
--epochs
2
--batch-size
512
--out
results-smoke
This checks the entire path with small inputs. It is too small to serve as the main accuracy experiment. On a laptop, use
--device cpu
for the same check. The script fails explicitly when
--device cuda
is requested but CUDA is unavailable.
Now the main run:
CUDA_VISIBLE_DEVICES
=
0 python
-u
train.py
--device
cuda
\
--rounds
2 3 4 5 6 22
--out
results
By default, this uses
1,048,576
training pairs,
65,536
validation pairs,
262,144
test pairs,
15
epochs, and batches of
8192
. These are starting parameters for the experiment. They are not a promise that this particular small architecture will distinguish every reduced-round configuration.
The raw keys, inputs, and labels are reused across round counts, so the comparison changes the encryption depth while keeping the sampled corpus fixed. The network is initialized and trained separately for each round count.
The program prints progress after each epoch and writes:
file
contents
metadata.json
arguments, versions, device name, source digest, completion status
metrics.csv
test measurements for both methods and each round count
rounds-N.pt
selected network weights, baseline weights, and experiment parameters
The CSV is flushed after each round-count experiment. The output directory must be new, so rerunning a command cannot silently replace the previous measurements. If the run is interrupted, its metadata remains
running
; use a different output directory for a fresh run.
Timing is synchronized on CUDA.
generation_seconds
covers tensor transfers, encryption, and feature construction for all three splits.
training_seconds
covers the neural training loop and validation. Neither field is an isolated cipher-throughput benchmark.
plotting the measurements
The plotting helper reads the CSV directly. It groups rows by method, plots their accuracy against round count, and uses the saved confidence bounds for error bars. The horizontal line at
0.5
marks chance performance on this balanced dataset.
Here is the complete helper (
plot.py
):
#!/usr/bin/env python3
"""Plot measured held-out accuracy; never synthesize benchmark values."""
import
argparse
import
csv
from
pathlib
import
Path
import
matplotlib
.
use
(
"Agg"
)
import
matplotlib.pyplot
as
plt
def
main
():
parser
=
argparse
.
ArgumentParser
(
description
=
__doc__
)
parser
.
add_argument
(
"csv"
,
type
=
Path
)
parser
.
add_argument
(
"--out"
,
type
=
Path
,
default
=
Path
(
"accuracy.png"
))
args
=
parser
.
parse_args
()
with
args
.
csv
.
open
(
newline
=
""
)
as
stream
:
rows
=
list
(
csv
.
DictReader
(
stream
))
if
not
rows
:
parser
.
error
(
"CSV contains no measurements"
)
fig
,
ax
=
plt
.
subplots
(
figsize
=
(
9
,
5
))
for
method
in
(
"bit_baseline"
,
"tiny_mlp"
):
selected
=
sorted
((
r
for
r
in
rows
if
r
[
"method"
]
==
method
),
key
=
lambda
r
:
int
(
r
[
"rounds"
]))
if
not
selected
:
continue
x
=
[
int
(
r
[
"rounds"
])
for
r
in
selected
]
y
=
[
float
(
r
[
"accuracy"
])
for
r
in
selected
]
errors
=
[[
max
(
0
,
a
-
float
(
r
[
"ci_low"
]))
for
a
,
r
in
zip
(
y
,
selected
)],
[
max
(
0
,
float
(
r
[
"ci_high"
])
-
a
)
for
a
,
r
in
zip
(
y
,
selected
)]]
ax
.
errorbar
(
x
,
y
,
yerr
=
errors
,
marker
=
"o"
,
capsize
=
4
,
label
=
f
"
{
method
}
(
{
selected
[
0
][
'features'
]
}
)"
)
ax
.
axhline
(
0.5
,
color
=
"gray"
,
linestyle
=
"--"
,
label
=
"balanced chance level"
)
ax
.
set
(
xlabel
=
"Speck32/64 rounds"
,
ylabel
=
"Held-out accuracy"
,
title
=
"Speck32/64: bit baseline and tiny MLP"
,
ylim
=
(
0
,
1.03
))
ax
.
set_xticks
(
sorted
({
int
(
r
[
"rounds"
])
for
r
in
rows
}))
ax
.
grid
(
alpha
=
0.2
)
ax
.
legend
()
fig
.
tight_layout
()
fig
.
savefig
(
args
.
out
,
dpi
=
180
)
print
(
f
"[=^..^=] wrote
{
args
.
out
}
"
)
if
__name__
==
"__main__"
:
main
()
Generate the figure:
python plot.py results/metrics.csv
--out
results/accuracy.png
The following measurements come from my
results/metrics.csv
: NVIDIA L20, seed
42
, and
262,144
held-out pairs with balanced classes. The MLP uses the complete ciphertext pair (
--features pair
), and labels are not shuffled. All values below are percentages:
rounds
bit baseline accuracy
tiny MLP accuracy
MLP false-positive rate
2
99.2260%
99.9985%
0.0031%
3
94.3920%
98.4596%
0.8224%
4
87.3127%
91.6782%
4.2625%
5
74.2729%
77.7096%
18.4517%
6
58.9195%
66.2334%
35.9184%
22
50.1610%
50.0004%
45.4231%
If the baseline wins, that is a result worth keeping. If the MLP stays near chance, the next question concerns the model, training budget, and representation. It does not automatically establish that the cipher distribution has no detectable structure.
controls that make the result useful
First, give the MLP the same output-difference bits available to the baseline:
CUDA_VISIBLE_DEVICES
=
0 python train.py
--features
xor
--out
results-xor
python plot.py results-xor/metrics.csv
--out
results-xor/accuracy.png
The default network sees the complete ciphertext pair, while the baseline sees only XOR differences. This control keeps the input representation equal and asks whether the learned nonlinear combinations help.
Second, independently shuffle the labels in all three splits:
CUDA_VISIBLE_DEVICES
=
0 python train.py
--rounds
2 5 22
\
--shuffle-labels
--out
results-shuffled
The permutation happens after encryption, independently in training, validation, and test. Both methods then learn and are evaluated against labels unrelated to the original input relationship. Consistently high held-out accuracy would be a reason to investigate leakage or an implementation error. Chance-level results in this control are expected.
Why also shuffle validation and test labels? With a small fixed training set, random labels can accidentally correlate with a strong feature. Selecting an epoch using the original validation labels can preserve that chance alignment. Independently shuffled test labels give us a cleaner check that unrelated labels do not generalize.
Third, repeat the main experiment with independent seeds:
for
seed
in
43 44
;
do
CUDA_VISIBLE_DEVICES
=
0 python train.py
--seed
"
$seed
"
--out
"results-seed-
$seed
"
done
Changing the seed changes both the sampled corpus and model initialization. Report the spread across these runs when discussing a small apparent improvement. Overlapping or non-overlapping per-model confidence intervals alone are not a complete comparison between the two classifiers, which make predictions on the same test examples.
The full
22
-round run is another control. A score near
0.5
means this experiment did not distinguish that setting with this data, model, and training budget. It is not a proof of cryptographic security.
practical example 3. a small local AI reads the results
The neural distinguisher above is the model that learns from encrypted blocks. We can add a separate, small LLM example: ask Qwen to summarize the measurements and identify their limits.
For this part, use:
ollama pull qwen2.5-coder:7b
The Ollama entry uses
Q4_K_M
quantization and has a download size of approximately
4.7 GB
. Runtime memory also includes the context and inference buffers.
Model card
.
The helper reads a completed
metrics.csv
, assigns an identifier such as
r5/tiny_mlp
to each row, and asks for up to three observations plus limitations. Every observation must cite a supplied row identifier.
For example, the schema requires an observation to contain these fields:
{
"row_id"
:
"an identifier from the measured CSV"
,
"text"
:
"an observation supported by that row"
}
That is a description of the format, not a fabricated model response. The request uses
temperature=0
, an
8192
-token context, and a JSON schema. Ollama’s
structured-output interface
constrains the response format. The helper checks the cited identifiers, but the interpretation still needs to be checked against the measurements.
We save the exact request and the raw response alongside the parsed report. That makes the AI part inspectable: the reader can see both what the model was told and what it actually returned.
Now the complete helper (
explain.py
):
#!/usr/bin/env python3
"""Optional local Qwen commentary on measured results. No model training."""
import
argparse
import
csv
import
json
from
pathlib
import
Path
import
urllib.error
import
urllib.request
def
main
():
parser
=
argparse
.
ArgumentParser
(
description
=
__doc__
)
parser
.
add_argument
(
"results"
,
type
=
Path
)
parser
.
add_argument
(
"--model"
,
default
=
"qwen2.5-coder:7b"
)
parser
.
add_argument
(
"--url"
,
default
=
"http://127.0.0.1:11434"
)
args
=
parser
.
parse_args
()
metadata
=
json
.
loads
((
args
.
results
/
"metadata.json"
).
read_text
())
if
metadata
.
get
(
"status"
)
!=
"complete"
:
parser
.
error
(
"run is incomplete; finish the experiment first"
)
with
(
args
.
results
/
"metrics.csv"
).
open
(
newline
=
""
)
as
stream
:
rows
=
list
(
csv
.
DictReader
(
stream
))
by_id
=
{
f
"r
{
r
[
'rounds'
]
}
/
{
r
[
'method'
]
}
"
:
r
for
r
in
rows
}
schema
=
{
"type"
:
"object"
,
"properties"
:
{
"observations"
:
{
"type"
:
"array"
,
"minItems"
:
1
,
"maxItems"
:
3
,
"items"
:
{
"type"
:
"object"
,
"properties"
:
{
"row_id"
:
{
"type"
:
"string"
,
"enum"
:
list
(
by_id
)},
"text"
:
{
"type"
:
"string"
}},
"required"
:
[
"row_id"
,
"text"
],
"additionalProperties"
:
False
}},
"limitations"
:
{
"type"
:
"array"
,
"minItems"
:
1
,
"maxItems"
:
3
,
"items"
:
{
"type"
:
"string"
}}},
"required"
:
[
"observations"
,
"limitations"
],
"additionalProperties"
:
False
}
context
=
(
"Summarize this synthetic Speck32/64 experiment in English. "
"Positive pairs have input XOR difference (0x0040,0); negative pairs "
"use independent uniform plaintexts under the same key within each pair. "
"Each sample has its own key; splits are key-disjoint and class-balanced. "
"The bit baseline uses independent XOR-bit likelihoods. The MLP uses "
"the features recorded in each row. Use only supplied measurements. "
"Cite a row_id for every observation. A Wilson interval concerns sampling "
"uncertainty for a fixed trained model, not variation across training runs. "
"An accuracy near 0.5 does not prove security. Distinguishing is not key "
"recovery, decryption, malware detection or an AV/EDR evaluation. "
"If shuffled_labels is True, labels were independently shuffled in all "
"three splits AFTER encryption; they no longer describe the input "
"relationship. Return JSON matching the schema."
)
body
=
dict
(
model
=
args
.
model
,
stream
=
False
,
keep_alive
=
0
,
format
=
schema
,
options
=
dict
(
temperature
=
0
,
seed
=
42
,
num_ctx
=
8192
,
num_predict
=
1200
),
messages
=
[
dict
(
role
=
"system"
,
content
=
context
),
dict
(
role
=
"user"
,
content
=
json
.
dumps
(
by_id
))])
(
args
.
results
/
"qwen-request.json"
).
write_text
(
json
.
dumps
(
body
,
indent
=
2
)
+
"
\n
"
)
request
=
urllib
.
request
.
Request
(
args
.
url
.
rstrip
(
"/"
)
+
"/api/chat"
,
data
=
json
.
dumps
(
body
).
encode
(),
headers
=
{
"Content-Type"
:
"application/json"
})
try
:
with
urllib
.
request
.
urlopen
(
request
,
timeout
=
300
)
as
response
:
result
=
json
.
load
(
response
)
(
args
.
results
/
"qwen-response.json"
).
write_text
(
json
.
dumps
(
result
,
indent
=
2
)
+
"
\n
"
)
if
not
result
.
get
(
"done"
)
or
result
.
get
(
"done_reason"
)
==
"length"
:
raise
ValueError
(
"model response is incomplete"
)
report
=
json
.
loads
(
result
[
"message"
][
"content"
])
if
not
isinstance
(
report
.
get
(
"observations"
),
list
)
or
not
report
[
"observations"
]:
raise
ValueError
(
"missing observations"
)
for
item
in
report
[
"observations"
]:
if
item
.
get
(
"row_id"
)
not
in
by_id
or
not
isinstance
(
item
.
get
(
"text"
),
str
):
raise
ValueError
(
"invalid observation or unknown evidence row"
)
if
not
isinstance
(
report
.
get
(
"limitations"
),
list
)
or
not
report
[
"limitations"
]:
raise
ValueError
(
"missing limitations"
)
if
not
all
(
isinstance
(
s
,
str
)
for
s
in
report
[
"limitations"
]):
raise
ValueError
(
"invalid limitations"
)
except
(
urllib
.
error
.
URLError
,
TimeoutError
,
ValueError
,
KeyError
,
TypeError
)
as
error
:
raise
SystemExit
(
f
"[=^..^=] Qwen report failed:
{
error
}
"
)
from
error
(
args
.
results
/
"qwen-report.json"
).
write_text
(
json
.
dumps
(
report
,
indent
=
2
)
+
"
\n
"
)
print
(
json
.
dumps
(
report
,
indent
=
2
))
if
__name__
==
"__main__"
:
main
()
demo 3
After the training experiment has finished, with a local Ollama service running:
python explain.py results
This creates
qwen-request.json
,
qwen-response.json
, and
qwen-report.json
. The script sends only the experiment rows to the local endpoint and does not execute anything returned by the model.
A note on Qwen’s mistakes.
As you can see, In this run, Qwen correctly copied the accuracy, TPR, and FPR values, but made two interpretation errors. Its phrase “2 rounds of training” refers to
two rounds of Speck encryption
; the MLP was trained for
15
epochs. It also described the baseline as trained for
0.06
seconds. That value comes from
generation_seconds
, which measures data transfers, encryption, and feature construction. The baseline’s
training_seconds
field is empty: its fitting time was not recorded. Valid JSON and correct CSV row references do not guarantee a correct explanation, so check the meaning of each field against the code.
If the server has several GPUs, Ollama’s GPU selection belongs on its server process. Setting
CUDA_VISIBLE_DEVICES
on the Python HTTP client does not move an already running Ollama service. For a separate instance on an unused port, one option is:
OLLAMA_HOST
=
127.0.0.1:11435
CUDA_VISIBLE_DEVICES
=
0 ollama serve
In another terminal:
OLLAMA_HOST
=
127.0.0.1:11435 ollama pull qwen2.5-coder:7b
python explain.py results
--url
http://127.0.0.1:11435
Again, choose the intended GPU index first. The helper asks Ollama to unload the model after the response, so use
ollama ps
on the corresponding service while the request is running if you want to inspect placement.
A useful report points to the measured difference between methods, mentions the negative controls, and identifies an unsupported conclusion. A response claiming that a distinguisher recovered a key or bypassed an EDR is wrong for this experiment. Keep that distinction visible when reviewing the output.
what this experiment tells us
The result is specific: for a selected cipher variant, input difference, and number of rounds, can a learned classifier distinguish related-input pairs from the independent-input control on unseen keys?
A positive answer demonstrates a statistical distinguisher in this experimental setting. Recovering an encryption key would require additional work and a separate attack analysis. A negative answer describes the limits of this classifier and budget.
There are also deliberate limits to the small implementation:
it uses a fixed training corpus and a small MLP; it does not reproduce the larger residual networks from the research paper;
keys are independently sampled per pair, so this evaluation does not by itself describe an attack against many observations under one fixed target key;
the script does not automatically find an unknown cipher, recover a key, or infer the number of rounds in an arbitrary executable;
the dataset contains no AV/EDR observations, so accuracy here cannot be translated into an evasion rate.
That last point connects back to the earlier malware cryptography posts. Cryptographic structure, static signatures, and runtime behavior are different measurements. A transformation can have useful diffusion while still being recognizable in a binary, and changing its statistical properties says nothing by itself about a security product’s verdict.
conclusion
This experiment connects a short C implementation to a measurable learning problem. We can inspect the cipher, verify its outputs, derive a baseline from bit probabilities, and train a network whose complete architecture fits in a few lines.
The interesting part is the comparison: how much structure does the baseline capture, what does the tiny network add, and which conclusions survive the controls? The screenshots and numbers should answer those questions after the L20 run.
I hope this post is useful for malware researchers, cryptography enthusiasts, red teamers, and blue team specialists.
Malware and cryptography 42: Speck
Malware and cryptography 38: S-box analyses
Malware and cryptography 45: Shamir Secret Sharing
The Simon and Speck Families of Lightweight Block Ciphers
Aron Gohr: Improving Attacks on Round-Reduced Speck32/64 Using Deep Learning
Research reference implementation
PyTorch
Qwen2.5-Coder 7B on Ollama
source code in github
This is a practical case for educational purposes only.
Thanks for your time, happy hacking and good bye!
PS. All drawings and screenshots are mine