Fabio/face-recognition-rk3588-opencl
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Initial validated RK3588 OpenCL face recognition engine

Browse source
This commit is contained in:
Fabio Micheluz 2026-10-01 12:43:56 +02:00
commit 8a731e2eb4
42 changed files with 16207 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
.gitignore vendored Normal file
View file

@ -0,0 +1,29 @@
# Build output
/build/
/bin/
# Runtime libraries and models
/lib/*.so
/lib/*.so.*
/models/*.rknn
# Local backups
*.before_*
*.before-*
*.bak
*~
# Runtime output
*.tmp
*.stderr
*.log
# Generated result files
*-cpu-raw.json
*-opencl-raw.json
*-cpp-cpu.json
*-cpp-opencl.json
# Editors
.vscode/
.idea/

99
Makefile Normal file
View file

@ -0,0 +1,99 @@
CXX ?= g++
ROOT := $(abspath .)
CXXFLAGS := \
-std=c++17 \
-O2 \
-Wall \
-Wextra \
-Wpedantic \
-Iinclude \
-DFACE_CLUSTERING_ROOT=\"$(ROOT)\"
LDFLAGS := \
-L$(ROOT)/lib \
-Wl,-rpath,'$$ORIGIN/../lib'
LDLIBS := \
-lrknnrt \
-ldl \
-lpthread
BIN_DIR := bin
BUILD_DIR := build
ENGINE_SOURCE := src/face_engine.cc
ENGINE_OBJECT := $(BUILD_DIR)/face_engine.o
FACE_RECOGNITION_SOURCE := src/face_recognition.cc
FACE_RECOGNITION_OBJECT := $(BUILD_DIR)/face_recognition.o
FACE_RECOGNITION_BINARY := $(BIN_DIR)/face_recognition
FACE_SCAN_SOURCE := src/face_scan.cc
FACE_SCAN_OBJECT := $(BUILD_DIR)/face_scan.o
FACE_SCAN_BINARY := $(BIN_DIR)/face_scan
.PHONY: all clean test info
all: \
$(FACE_RECOGNITION_BINARY) \
$(FACE_SCAN_BINARY)
$(BUILD_DIR):
mkdir -p $(BUILD_DIR)
$(BIN_DIR):
mkdir -p $(BIN_DIR)
$(ENGINE_OBJECT): $(ENGINE_SOURCE) include/face_engine.h | $(BUILD_DIR)
$(CXX) $(CXXFLAGS) -c $< -o $@
$(FACE_RECOGNITION_OBJECT): \
$(FACE_RECOGNITION_SOURCE) \
include/face_engine.h \
include/face_config.h \
| $(BUILD_DIR)
$(CXX) $(CXXFLAGS) -c $< -o $@
$(FACE_SCAN_OBJECT): \
$(FACE_SCAN_SOURCE) \
include/face_engine.h \
include/face_config.h \
| $(BUILD_DIR)
$(CXX) $(CXXFLAGS) -c $< -o $@
$(FACE_RECOGNITION_BINARY): \
$(ENGINE_OBJECT) \
$(FACE_RECOGNITION_OBJECT) \
| $(BIN_DIR)
$(CXX) $^ $(LDFLAGS) $(LDLIBS) -o $@
$(FACE_SCAN_BINARY): \
$(ENGINE_OBJECT) \
$(FACE_SCAN_OBJECT) \
| $(BIN_DIR)
$(CXX) $^ $(LDFLAGS) $(LDLIBS) -o $@
test: all
./bin/face_recognition \
test/test.jpg \
test/test3f.jpg
./bin/face_scan \
test/test3f.jpg \
> /tmp/face_scan_test.json
python3 -m json.tool \
/tmp/face_scan_test.json \
> /dev/null
@echo "Tests completed successfully"
info:
@echo "ROOT=$(ROOT)"
@echo "CXX=$(CXX)"
@echo "CXXFLAGS=$(CXXFLAGS)"
@echo "LDFLAGS=$(LDFLAGS)"
clean:
rm -rf $(BUILD_DIR)
rm -f $(FACE_RECOGNITION_BINARY)
rm -f $(FACE_SCAN_BINARY)

481
README.md Normal file
View file

@ -0,0 +1,481 @@
# Face Recognition RK3588 OpenCL
Motore C++ per rilevamento facciale, generazione di embedding e riconoscimento kNN accelerato su Rockchip RK3588.
Il progetto usa:
- NPU tramite RKNN per SCRFD e SFace;
- GPU Mali-G610 tramite OpenCL per kNN;
- CPU C++ come riferimento numerico e fallback.
## Funzionalità
- rilevamento volti con SCRFD;
- confidence, bounding box e cinque landmark;
- non-maximum suppression;
- allineamento volto 112 x 112;
- embedding SFace float32 a 128 dimensioni;
- normalizzazione L2;
- cosine similarity tramite prodotto scalare;
- kNN CPU e OpenCL;
- conteggi dinamici, TOPK configurabile e chunking;
- leave-one-out tramite self-map;
- JSON su stdout e diagnostica su stderr.
## Architettura
```text
Immagine
|
v
SCRFD tramite RKNN / NPU
|
v
Bounding box e landmark
|
v
Allineamento volto 112 x 112
|
v
SFace tramite RKNN / NPU
|
v
Embedding float32[128] normalizzato L2
|
+-----------------------------+
| |
v v
kNN CPU C++ kNN OpenCL / Mali-G610
| |
+-------------+---------------+
|
v
TOPK prototype
|
v
Leave-one-out opzionale
|
v
Soglia di riconoscimento
```
## Formato degli embedding
```text
Dimensione: 128
Tipo: float32
Dimensione binaria: 512 byte
Ordine byte: little-endian
Normalizzazione: L2
Norma attesa: circa 1.0
```
Poiché gli embedding sono normalizzati:
```text
cosine_similarity(a, b) = dot(a, b)
```
## Componenti
### Face engine RKNN
```text
include/face_config.h
include/face_engine.h
include/rknn_api.h
src/face_engine.cc
src/face_recognition.cc
src/face_scan.cc
```
### Motore kNN
```text
include/knn_opencl.h
src/knn_opencl.cc
src/knn_opencl_backend.cc
src/face_knn.cc
kernels/knn_float4_topk.cl
kernels/knn_reference.cl
```
Il kernel produttivo viene compilato a runtime, ad esempio con:
```text
-cl-fast-relaxed-math
-D DIM=128
-D TOPK=5
-D QUERY_COUNT=21
-D LANES_PER_QUERY=8
```
## Requisiti
- Linux AArch64;
- Rockchip RK3588 o piattaforma compatibile;
- compilatore C++17 e GNU Make;
- runtime RKNN;
- OpenCL ICD loader e driver Mali;
- modelli SCRFD e SFace in formato RKNN.
Su Debian:
```bash
sudo apt-get install \
build-essential \
opencl-headers \
ocl-icd-opencl-dev
```
Verifica OpenCL:
```bash
clinfo 2>/dev/null \
| grep -E 'Platform Name|Device Name|Device Type|Device Version'
```
## File locali richiesti
Non sono versionati:
```text
models/SCRFD_500M_KPS_640.rknn
models/face_recognition_sface_2021dec.rknn
lib/librknnrt.so
```
## Compilazione
Dalla root del repository:
```bash
make clean
make -j"$(nproc)"
```
Binari previsti:
```text
bin/face_recognition
bin/face_scan
bin/face_knn
```
Verifica del binario OpenCL:
```bash
file bin/face_knn
ldd bin/face_knn | grep -Ei 'opencl|stdc|not found'
```
## Utilizzo dei binari
### face_recognition
Programma dimostrativo con output destinato alla lettura umana.
Analisi di una singola fotografia:
```bash
./bin/face_recognition test/test.jpg
```
Confronto tra due fotografie:
```bash
./bin/face_recognition \
test/test.jpg \
test/test3f.jpg
```
Usarlo per verificare modelli e runtime RKNN, detection, landmark, embedding e similarità. Non è il binario consigliato per integrazioni applicative.
### face_scan
Elabora una fotografia e produce JSON con immagine, volti, confidence, bounding box, landmark ed embedding.
```bash
./bin/face_scan test/test3f.jpg
```
Separazione di risultato e diagnostica:
```bash
./bin/face_scan test/test3f.jpg \
> /tmp/face-scan.json \
2> /tmp/face-scan.stderr
```
Validazione:
```bash
python3 -m json.tool /tmp/face-scan.json > /dev/null \
&& echo "OK: JSON valido"
```
Exit code:
```text
0 scansione completata
2 argomenti non validi
3 inizializzazione FaceEngine fallita
4 caricamento o elaborazione immagine fallita
5 risultato facciale o embedding non valido
```
Usarlo per integrare il face engine con Node.js, Python o altri linguaggi e per costruire query destinate a `face_knn`.
### face_knn
Confronta embedding query con un database di prototype. Backend disponibili:
```text
cpu
opencl
```
Il programma va normalmente eseguito dalla root del repository, perché il kernel viene caricato da:
```text
kernels/knn_float4_topk.cl
```
Help:
```bash
./bin/face_knn --help
```
Backend CPU:
```bash
./bin/face_knn \
--queries queries.bin \
--database prototypes.bin \
--query-count 21 \
--database-count 5 \
--dimension 128 \
--top-k 5 \
--threshold 0.50 \
--backend cpu
```
Backend OpenCL:
```bash
./bin/face_knn \
--queries queries.bin \
--database prototypes.bin \
--query-count 21 \
--database-count 5 \
--dimension 128 \
--top-k 5 \
--threshold 0.50 \
--backend opencl
```
Argomenti:
```text
--queries file binario degli embedding query
--database file binario degli embedding prototype
--query-count numero di query
--database-count numero di prototype
--dimension componenti per embedding, 128 per SFace
--top-k numero massimo di vicini
--threshold soglia minima di riconoscimento
--self-map esclusioni leave-one-out opzionali
--backend cpu oppure opencl
```
Il TOPK effettivo è:
```text
effective_top_k = min(top_k, database_count)
```
#### Leave-one-out
Quando una query è anche un prototype, il confronto con sé stessa va escluso. La self-map contiene una riga per query:
```text
-1
1
3
0
4
-1
```
`-1` indica nessuna esclusione; un valore da zero in su indica l'indice del prototype da escludere.
```bash
./bin/face_knn \
--queries queries.bin \
--database prototypes.bin \
--query-count 21 \
--database-count 5 \
--dimension 128 \
--top-k 5 \
--threshold 0.50 \
--self-map self-map.txt \
--backend opencl
```
#### Output e diagnostica
```bash
./bin/face_knn \
--queries queries.bin \
--database prototypes.bin \
--query-count 21 \
--database-count 5 \
--dimension 128 \
--top-k 5 \
--threshold 0.50 \
--self-map self-map.txt \
--backend opencl \
> /tmp/face-knn.json \
2> /tmp/face-knn.stderr
```
```text
stdout → JSON valido
stderr → diagnostica OpenCL e driver Mali
```
Validazione:
```bash
python3 -m json.tool /tmp/face-knn.json > /dev/null \
&& echo "OK: JSON valido"
```
Campi principali del JSON:
```text
backend, device_name, query_count, database_count, dimension, top_k,
threshold, leave_one_out, elapsed_ms, results, recognized_count,
below_threshold_count, coverage_percent
```
Per ogni query:
```text
query_index, excluded_index, recognized, best_index, best_score, neighbors
```
### test_knn_cpu
Sorgente:
```text
test/knn/test_knn_cpu.cc
```
Esecuzione, se compilato:
```bash
./bin/test_knn_cpu
```
Output atteso:
```text
OK: backend CPU dinamico
query_count=2
database_count=3
top_k=2
```
## Scelta del backend
Usare CPU per database piccoli, riferimento numerico o fallback. Usare OpenCL per database grandi o molte query. Con cinque prototype l'overhead OpenCL può superare il costo del calcolo; con 350000 prototype e 256 query è stato misurato uno speedup di circa 7 volte.
## Flusso completo
```text
face_scan
→ estrae gli embedding
→ applicazione seleziona i prototype
→ face_knn calcola TOPK
→ applicazione associa gli indici alle identità
→ applicazione applica soglia e margine
→ applicazione salva il risultato
```
## Prestazioni sintetiche validate
```text
Database: 350000
Query: 256
Dimensione: 128
TOPK: 32
Chunk: 50000
GPU: Mali-G610 r0p0
CPU: 9663.226 ms
OpenCL: 1373.797 ms
Speedup: 7.034x
TOPK identico: 8192 / 8192
```
Variante float4 min-replace:
```text
Kernel: 1244.576 ms
Throughput: 71.992 M vector/s
Dot rate: 18.430 GFLOP/s
Traffico effettivo: 36.860 GB/s
```
## Validazione reale
```text
Query: 21
Prototype: 5
Dimensione: 128
Soglia: 0.50
Leave-one-out: attivo
Riconosciuti: 20 / 21
Copertura: 95.2%
Caso sotto soglia: IMG_0183.JPG
Score: 0.466428
Miglior prototype: IMG_0143.JPG
```
Confronti:
```text
Python vs C++ CPU: 21 / 21 query coerenti
CPU vs OpenCL dopo leave-one-out: 100 / 100 posizioni
CPU vs OpenCL TOPK grezzo: 105 / 105 posizioni
Errore massimo CPU/OpenCL: 3.58e-7
```
Dettagli:
```text
VALIDATION_JESSICA.md
README1.md
modules/knn-opencl/reference/VALIDATION.md
```
## Integrazione applicativa
Un'applicazione deve leggere e validare i prototype, generare i file binari e i metadati, invocare `face_knn`, associare gli indici alle identità, applicare soglia e margine e salvare il risultato. La prima integrazione dovrebbe funzionare in modalità shadow, confrontando il risultato corrente con OpenCL prima dell'attivazione definitiva.
## Stato
```text
Face engine RKNN: validato
Backend CPU: validato
Backend OpenCL: validato
Mali-G610: validata
TOPK dinamico: validato
Chunking: validato
Leave-one-out: validato
Output JSON puro: validato
Dataset reale: validato
Integrazione applicativa: da completare
```

239
README1.md Normal file
View file

@ -0,0 +1,239 @@
# Note tecniche e cronologia del kNN OpenCL
Questo documento raccoglie il percorso sperimentale che ha portato al motore kNN OpenCL. Per installazione, compilazione e uso dei binari consultare `README.md`.
## Obiettivo
Input:
```text
float32[N][128]
```
Gli embedding sono normalizzati L2, quindi cosine similarity e prodotto scalare coincidono. Output:
```text
indices[query_count][TOPK]
scores[query_count][TOPK]
```
## Baseline
```text
DIM=128
TOPK=32
QUERIES=256
LANES_PER_QUERY=8
CHUNK_SIZE=50000
```
## Risultati sperimentali principali
### Numero di lane
```text
LANES=4: 9.60 GOP/s
LANES=8: 10.57 GOP/s
LANES=16: 4.04 GOP/s
LANES=32: 1.49 GOP/s
```
Otto lane sono risultate la configurazione più equilibrata.
### Stage benchmark
```text
DOT: 72.79 M vector/s
DOT+threshold: 83.47 M vector/s
TOPK score: 77.94 M vector/s
TOPK+indici: 76.28 M vector/s
kNN completo: 75.22 M vector/s
```
### Vettorizzazione
```text
scalar: 74.37 M vector/s
float4: 84.14 M vector/s
float8: 83.49 M vector/s
```
Database e query entrambi float4 hanno raggiunto 84.13 M vector/s.
### Accumulatori
```text
F4 singolo accumulatore: 74.53 M vector/s
F4 quattro accumulatori: 83.47 M vector/s
F4 quattro acc + unroll2: 70.53 M vector/s
F8 quattro accumulatori: 82.76 M vector/s
```
L'unrolling esplicito ha peggiorato il code generation del compilatore Mali.
### Pressione sui registri
Una query mantenuta in `private float4 query[DIM4]` ha ridotto le prestazioni a 17.85 M vector/s. Rimuovendo l'array privato si è risaliti a circa 65 M vector/s.
### Strategie TOPK
```text
sorted insertion: 78.37 M vector/s
min-replace: 81.99 M vector/s
threshold + min-replace: 82.02 M vector/s
```
### Dimensione TOPK
```text
TOPK=8: 73.87 M vector/s
TOPK=16: 84.60 M vector/s
TOPK=32: 82.53 M vector/s
TOPK=64: 78.81 M vector/s
```
## Benchmark da 350000 embedding
```text
Database: 350000
Query: 256
Dimensione: 128
TOPK: 32
Chunk: 50000
Confronti: 89600000
```
Variante min-replace:
```text
KNN kernel: 1244.576 ms
Merge GPU: 8.022 ms
Merge CPU globale: 14.039 ms
Tempo complessivo: 1266.638 ms
Vector rate: 71.992 M vector/s
Dot rate: 18.430 GFLOP/s
Traffico: 36.860 GB/s
Indici non validi: 0
```
Benchmark parametrico:
```text
CPU: 9663.226 ms
OpenCL: 1373.797 ms
Speedup: 7.034x
Posizioni TOPK identiche: 8192 / 8192
Exact ratio: 100%
```
## Trasformazione in motore dinamico
Il kernel produttivo usa definizioni configurabili:
```c
#ifndef DIM
#define DIM 128
#endif
#ifndef TOPK
#define TOPK 32
#endif
#ifndef QUERY_COUNT
#define QUERY_COUNT 1
#endif
#ifndef LANES_PER_QUERY
#define LANES_PER_QUERY 8
#endif
```
Il limite fisso `q >= 256` è diventato `q >= QUERY_COUNT`. Le definizioni vengono passate a `clBuildProgram` insieme a `-cl-fast-relaxed-math`.
## Backend CPU
È stato generalizzato per conteggi e dimensione dinamici, TOPK sicuro, ordinamento deterministico, rifiuto di valori non finiti e layout identico a OpenCL.
## Backend OpenCL
Il backend:
1. enumera piattaforme e GPU;
2. seleziona preferibilmente Mali-G610;
3. carica il kernel da file;
4. compila con parametri runtime;
5. alloca buffer riutilizzabili;
6. copia il database a chunk;
7. esegue il kernel;
8. legge indici e score;
9. converte indici locali in globali;
10. effettua il merge TOPK;
11. rilascia le risorse tramite RAII.
## Diagnostica Mali
Il runtime Mali emette `arm_release_ver` direttamente su stdout. La CLI devia temporaneamente stdout verso stderr durante OpenCL e lo ripristina prima della scrittura JSON.
## Validazione reale
Prototype:
```text
0 IMG_0145.JPG cluster_medoid
1 IMG_0141.JPG cluster_medoid
2 IMG_0193.JPG cluster_medoid
3 IMG_0143.JPG greedy_leave_one_out_gain
4 IMG_0146.JPG greedy_leave_one_out_gain
```
Risultato:
```text
Query: 21
Prototype: 5
Riconosciuti: 20
Sotto soglia: 1
Copertura: 95.2%
Caso sotto soglia: IMG_0183.JPG
Score: 0.466428
Miglior prototype: IMG_0143.JPG
```
Python contro C++ CPU:
```text
Query confrontate: 21
Mismatch top-1: 0
Mismatch decisione: 0
Errore massimo: 2.643e-7
```
C++ CPU contro OpenCL:
```text
Dopo leave-one-out: 100 / 100 posizioni identiche
TOPK grezzo: 105 / 105 posizioni identiche
Errore massimo: 3.58e-7
```
## File di riferimento
```text
modules/knn-opencl/reference/knn_benchmark.cpp
modules/knn-opencl/reference/knn_float4_minreplace_validate_benchmark.cpp
modules/knn-opencl/reference/validation-opencl-350k.txt
modules/knn-opencl/reference/validation-minreplace-350k.txt
modules/knn-opencl/reference/VALIDATION.md
VALIDATION_JESSICA.md
```
## Sviluppi futuri
```text
backend auto
cache del programma OpenCL
indice persistente dei prototype
margine fra prima e seconda identità
modalità shadow applicativa
metriche operative
```

9
SHA256SUMS Normal file
View file

@ -0,0 +1,9 @@
9fb33ec13abd1714832d6bc81a3b3df553d80577195e54ff8bfdbc33220a6566 README.md
7f9373326a666d36dcd26fcf9d2326fa56198b5aab448d53651fe5aff379be57 README1.md
c6b144c487af817ea1578f3190e968d81a09dcee5d4cdc9f8f5e664b6cd922b5 VALIDATION_JESSICA.md
f0f057dda1dd0bb8d3e097698da7b8ea956a01217b47349a99936729aefd2f3d include/knn_opencl.h
ca211d9e68b4c15b1fcbf8a2607ba398b845f7fab7db417302642f5ed2e0413e src/knn_opencl.cc
918bed774c805cafb2645214ab239852b96ed4eb104fcdf78ac2488d64c19737 src/knn_opencl_backend.cc
4561c83dcec5015e09ee5e7990698c50eed9d2251dc56349fe51cbc579d011a3 src/face_knn.cc
2721831e9cb285c3ef12aedb59fb66ca35e067321f29e67582a39e58e47d283b kernels/knn_reference.cl
6c4f502438aad3dd1600a00cb7988e0e35e56b2e8e4620d44ffcb0f3610a7226 kernels/knn_float4_topk.cl

49
VALIDATION_JESSICA.md Normal file
View file

@ -0,0 +1,49 @@
# Validazione kNN reale
## Dataset
- Persona: Jessica
- Query: 21
- Prototype: 5
- Dimensione embedding: 128 float32
- Soglia: 0.50
- Leave-one-out: attivo
## Python vs C++ CPU
- Query confrontate: 21
- Mismatch top-1: 0
- Mismatch decisione: 0
- Errore massimo score: 2.643e-7
## C++ CPU vs OpenCL dopo leave-one-out
- Posizioni confrontate: 100
- Indici identici: 100
- Exact ratio: 100%
- Mismatch decisione: 0
- Errore massimo score: 3.58e-7
## C++ CPU vs OpenCL TOPK grezzo
- Posizioni confrontate: 105
- Indici identici: 105
- Exact ratio: 100%
- Mismatch score oltre 1e-5: 0
- Errore massimo score: 3.58e-7
## Risultato riconoscimento
- Riconosciuti: 20/21
- Copertura: 95.2%
- Unico caso sotto soglia: IMG_0183.JPG
- Best similarity: 0.466428
- Miglior prototype: IMG_0143.JPG
## Backend OpenCL
- Device: ARM / Mali-G610 r0p0
- DIM: 128
- TOPK: 5
- QUERY_COUNT: 21
- LANES_PER_QUERY: 8

144
docs/architecture.md Normal file
View file

@ -0,0 +1,144 @@
# Architettura
Il progetto separa il motore di elaborazione facciale dai programmi che lo
utilizzano.
## Componenti principali
### face_engine.h
Definisce l'interfaccia pubblica C++:
- `Point`;
- `Face`;
- `FaceEngine`;
- inizializzazione dei modelli;
- elaborazione di una singola immagine.
### face_engine.cc
Implementa:
- caricamento delle immagini tramite stb_image;
- resize bilineare;
- conversione float32 e float16;
- inferenza SCRFD tramite RKNN;
- decodifica di score, bounding box e landmark;
- non-maximum suppression;
- trasformazione similarity;
- allineamento facciale;
- inferenza SFace tramite RKNN;
- conversione dell'output;
- normalizzazione L2 dell'embedding.
### face_recognition.cc
Programma dimostrativo con output leggibile.
Può:
- elaborare una singola immagine;
- elaborare due immagini;
- confrontare tutti i volti rilevati tramite cosine similarity.
### face_scan.cc
Programma destinato all'integrazione con altri software.
Caratteristiche:
- elabora una singola immagine;
- scrive esclusivamente JSON su standard output;
- scrive diagnostica ed errori su standard error;
- valida la dimensione degli embedding;
- rifiuta valori NaN o infiniti;
- restituisce exit code documentati.
## Pipeline runtime
```text
File immagine
|
v
Caricamento RGB
|
v
Resize proporzionale
|
v
Input SCRFD 640 x 640 FLOAT16 NHWC
|
v
SCRFD su NPU
|
v
Decodifica livelli stride 8, 16 e 32
|
v
Score threshold
|
v
Non-maximum suppression
|
v
Bounding box e cinque landmark
|
v
Similarity transform
|
v
Volto allineato 112 x 112
|
v
SFace su NPU
|
v
Embedding 128D
|
v
Normalizzazione L2
```
## Riutilizzo
Le applicazioni C++ possono utilizzare direttamente `FaceEngine`.
Applicazioni scritte in altri linguaggi possono eseguire
`face_scan` come processo figlio e analizzare il JSON prodotto.
Il motore non dipende:
- dal server della galleria;
- dal database;
- dall'interfaccia utente;
- da uno specifico framework applicativo.
## Estensione futura
Gli embedding prodotti da `FaceEngine` saranno utilizzati dalla
pipeline di clustering:
```text
FaceEngine
|
v
Embedding float32[N][128]
|
v
kNN OpenCL TOPK=32
|
v
Core distance
|
v
Mutual reachability
|
v
Minimum spanning tree
|
v
HDBSCAN
|
v
Cluster ID per embedding
```

113
docs/json-format.md Normal file
View file

@ -0,0 +1,113 @@
# Formato JSON di face_scan
`face_scan` scrive un singolo oggetto JSON su standard output.
Gli errori e i messaggi diagnostici vengono scritti su standard error.
## Invocazione
```bash
./bin/face_scan /percorso/immagine.jpg
```
## Esempio
```json
{
"image": {
"path": "/percorso/immagine.jpg",
"width": 1500,
"height": 2000
},
"faces": [
{
"face_index": 0,
"score": 0.8671875,
"bbox": {
"x": 1079.3457,
"y": 489.648438,
"width": 131.591797,
"height": 182.275391
},
"landmarks": [
[1107.8125, 558.325195],
[1170.99609, 559.423828],
[1134.17969, 594.500732],
[1106.22559, 615.83252],
[1168.34717, 616.967773]
],
"embedding": [
-0.0426222458,
0.0678814054
]
}
]
}
```
L'array `embedding` reale contiene sempre 128 valori.
## Oggetto image
Campi disponibili:
- `path`: percorso ricevuto dal programma;
- `width`: larghezza originale dell'immagine;
- `height`: altezza originale dell'immagine.
## Oggetto face
Campi disponibili:
- `face_index`: indice sequenziale del volto;
- `score`: confidence SCRFD;
- `bbox.x`: coordinata sinistra;
- `bbox.y`: coordinata superiore;
- `bbox.width`: larghezza;
- `bbox.height`: altezza;
- `landmarks`: cinque coppie di coordinate;
- `embedding`: vettore L2-normalizzato di 128 elementi.
## Immagini senza volti
Un'immagine elaborata correttamente ma senza volti produce:
```json
{
"image": {
"path": "image.jpg",
"width": 1920,
"height": 1080
},
"faces": []
}
```
L'exit code resta `0`.
## Exit code
```text
0 scansione completata
2 argomenti non validi
3 inizializzazione FaceEngine fallita
4 caricamento o elaborazione immagine fallita
5 risultato facciale o embedding non valido
```
## Requisiti dell'embedding
Per ogni volto:
```text
Dimensione 128
Tipo logico float32
Norma L2 circa 1.0
Valori tutti finiti
```
Quando viene salvato in formato binario float32:
```text
128 x 4 byte = 512 byte
```

5
include/face_config.h Normal file
View file

@ -0,0 +1,5 @@
#pragma once
#ifndef FACE_CLUSTERING_ROOT
#define FACE_CLUSTERING_ROOT "."
#endif

47
include/face_engine.h Normal file
View file

@ -0,0 +1,47 @@
#pragma once
#include <string>
#include <vector>
struct Point {
float x = 0.0f;
float y = 0.0f;
};
struct Face {
float score = 0.0f;
float x1 = 0.0f;
float y1 = 0.0f;
float x2 = 0.0f;
float y2 = 0.0f;
Point kps[5]{};
// SFace: embedding L2-normalizzato da 128 float32.
std::vector<float> embedding;
};
class FaceEngine {
public:
FaceEngine();
~FaceEngine();
FaceEngine(const FaceEngine&) = delete;
FaceEngine& operator=(const FaceEngine&) = delete;
bool init(
const std::string& scrfd_model,
const std::string& sface_model);
bool process_image(
const std::string& image_path,
std::vector<Face>& faces,
int& image_width,
int& image_height,
int rotation_degrees = 0);
private:
struct Impl;
Impl* impl_;
};

121
include/knn_opencl.h Normal file
View file

@ -0,0 +1,121 @@
#ifndef FACE_CLUSTERING_KNN_OPENCL_H
#define FACE_CLUSTERING_KNN_OPENCL_H
#include <cstddef>
#include <cstdint>
#include <string>
#include <vector>
namespace face_knn {
constexpr std::uint32_t invalid_index =
0xffffffffu;
struct Neighbor {
float score = 0.0f;
std::uint32_t index =
invalid_index;
};
struct SearchOptions {
std::uint32_t dimension = 128;
std::uint32_t top_k = 1;
std::uint32_t lanes_per_query = 8;
std::size_t chunk_size = 50000;
std::string kernel_path;
};
struct SearchOutput {
std::size_t query_count = 0;
std::size_t database_count = 0;
std::uint32_t dimension = 0;
std::uint32_t top_k = 0;
std::string backend;
std::string device_name;
double elapsed_ms = 0.0;
/*
* Layout:
*
* neighbors[
* query_index * top_k
* + neighbor_position
* ]
*/
std::vector<Neighbor> neighbors;
};
struct ValidationResult {
std::size_t positions = 0;
std::size_t exact_index_matches = 0;
std::size_t decision_matches = 0;
float maximum_score_error = 0.0f;
bool exact_indices = false;
bool decisions_equal = false;
};
void validate_input(
const std::vector<float>& database,
const std::vector<float>& queries,
std::size_t database_count,
std::size_t query_count,
const SearchOptions& options);
SearchOutput search_cpu(
const std::vector<float>& database,
const std::vector<float>& queries,
std::size_t database_count,
std::size_t query_count,
const SearchOptions& options);
SearchOutput search_opencl(
const std::vector<float>& database,
const std::vector<float>& queries,
std::size_t database_count,
std::size_t query_count,
const SearchOptions& options);
ValidationResult compare_outputs(
const SearchOutput& expected,
const SearchOutput& actual,
float score_tolerance,
float recognition_threshold);
std::vector<Neighbor> query_neighbors(
const SearchOutput& output,
std::size_t query_index);
std::vector<Neighbor> exclude_neighbor(
const std::vector<Neighbor>& neighbors,
std::uint32_t excluded_index);
} // namespace face_knn
#endif

804
include/rknn_api.h Normal file
View file

@ -0,0 +1,804 @@
/****************************************************************************
*
* Copyright (c) 2017 - 2022 by Rockchip Corp. All rights reserved.
*
* The material in this file is confidential and contains trade secrets
* of Rockchip Corporation. This is proprietary information owned by
* Rockchip Corporation. No part of this work may be disclosed,
* reproduced, copied, transmitted, or used in any way for any purpose,
* without the express written permission of Rockchip Corporation.
*
*****************************************************************************/
#ifndef _RKNN_API_H
#define _RKNN_API_H
#ifdef __cplusplus
extern "C" {
#endif
#include <stdint.h>
/*
Definition of extended flag for rknn_init.
*/
/* set high priority context. */
#define RKNN_FLAG_PRIOR_HIGH 0x00000000
/* set medium priority context */
#define RKNN_FLAG_PRIOR_MEDIUM 0x00000001
/* set low priority context. */
#define RKNN_FLAG_PRIOR_LOW 0x00000002
/* asynchronous mode.
when enable, rknn_outputs_get will not block for too long because it directly retrieves the result of
the previous frame which can increase the frame rate on single-threaded mode, but at the cost of
rknn_outputs_get not retrieves the result of the current frame.
in multi-threaded mode you do not need to turn this mode on. */
#define RKNN_FLAG_ASYNC_MASK 0x00000004
/* collect performance mode.
when enable, you can get detailed performance reports via rknn_query(ctx, RKNN_QUERY_PERF_DETAIL, ...),
but it will reduce the frame rate. */
#define RKNN_FLAG_COLLECT_PERF_MASK 0x00000008
/* allocate all memory in outside, includes weight/internal/inputs/outputs */
#define RKNN_FLAG_MEM_ALLOC_OUTSIDE 0x00000010
/* weight sharing with the same network structure */
#define RKNN_FLAG_SHARE_WEIGHT_MEM 0x00000020
/* send fence fd from outside */
#define RKNN_FLAG_FENCE_IN_OUTSIDE 0x00000040
/* get fence fd from inside */
#define RKNN_FLAG_FENCE_OUT_OUTSIDE 0x00000080
/* dummy init flag: could only get total_weight_size and total_internal_size by rknn_query*/
#define RKNN_FLAG_COLLECT_MODEL_INFO_ONLY 0x00000100
/* allocate internal memory in outside */
#define RKNN_FLAG_INTERNAL_ALLOC_OUTSIDE 0x00000200
/* set GPU as the preferred execution backend When the operator is not supported by the NPU */
#define RKNN_FLAG_EXECUTE_FALLBACK_PRIOR_DEVICE_GPU 0x00000400
/* enable allocate sram type buffers */
#define RKNN_FLAG_ENABLE_SRAM 0x00000800
/* sram type buffers are shared among different contexts */
#define RKNN_FLAG_SHARE_SRAM 0x00001000
/* default nice -19, this flag can disable default priority */
#define RKNN_FLAG_DISABLE_PROC_HIGH_PRIORITY 0x00002000
/* don't flush input buffer cache, the user must ensure that the input tensor has flushed the cache before calling rknn_run.
!!! Don't use this flags when you call rknn_inputs_set() to set input data. */
#define RKNN_FLAG_DISABLE_FLUSH_INPUT_MEM_CACHE 0x00004000
/* Don't invalid output buffer cache.
Users cannot directly access output_mem->virt_addr,
which will cause cache consistency problems.
If you want to use output_mem->virt_addr,
you must use rknn_mem_sync (ctx, mem, RKNN_MEMORY_SYNC_FROM_DEVICE) to flush the cache.
This flags is generally used when the output data of the NPU is not accessed by the CPU,
but is accessed by the GPU or RGA to reduce the time required to flush the cache.
!!! Don't use this flags when you call rknn_outputs_get() to get output data.*/
#define RKNN_FLAG_DISABLE_FLUSH_OUTPUT_MEM_CACHE 0x00008000
/* This flag is used when the model data buffer is allocated by NPU, and can be accessed by NPU directly. */
#define RKNN_FLAG_MODEL_BUFFER_ZERO_COPY 0x00010000
/* This flag is a memory allocation flag, which is used in rknn_create_mem2() when no context is available. */
#define RKNN_MEM_FLAG_ALLOC_NO_CONTEXT 0x00020000
/*
Error code returned by the RKNN API.
*/
#define RKNN_SUCC 0 /* execute succeed. */
#define RKNN_ERR_FAIL -1 /* execute failed. */
#define RKNN_ERR_TIMEOUT -2 /* execute timeout. */
#define RKNN_ERR_DEVICE_UNAVAILABLE -3 /* device is unavailable. */
#define RKNN_ERR_MALLOC_FAIL -4 /* memory malloc fail. */
#define RKNN_ERR_PARAM_INVALID -5 /* parameter is invalid. */
#define RKNN_ERR_MODEL_INVALID -6 /* model is invalid. */
#define RKNN_ERR_CTX_INVALID -7 /* context is invalid. */
#define RKNN_ERR_INPUT_INVALID -8 /* input is invalid. */
#define RKNN_ERR_OUTPUT_INVALID -9 /* output is invalid. */
#define RKNN_ERR_DEVICE_UNMATCH -10 /* the device is unmatch, please update rknn sdk
and npu driver/firmware. */
#define RKNN_ERR_INCOMPATILE_PRE_COMPILE_MODEL -11 /* This RKNN model use pre_compile mode, but not compatible with current driver. */
#define RKNN_ERR_INCOMPATILE_OPTIMIZATION_LEVEL_VERSION -12 /* This RKNN model set optimization level, but not compatible with current driver. */
#define RKNN_ERR_TARGET_PLATFORM_UNMATCH -13 /* This RKNN model set target platform, but not compatible with current platform. */
/*
Definition for tensor
*/
#define RKNN_MAX_DIMS 16 /* maximum dimension of tensor. */
#define RKNN_MAX_NUM_CHANNEL 15 /* maximum channel number of input tensor. */
#define RKNN_MAX_NAME_LEN 256 /* maximum name lenth of tensor. */
#define RKNN_MAX_DYNAMIC_SHAPE_NUM 512 /* maximum number of dynamic shape for each input. */
#ifdef __arm__
typedef uint32_t rknn_context;
#else
typedef uint64_t rknn_context;
#endif
/*
The query command for rknn_query
*/
typedef enum _rknn_query_cmd {
RKNN_QUERY_IN_OUT_NUM = 0, /* query the number of input & output tensor. */
RKNN_QUERY_INPUT_ATTR = 1, /* query the attribute of input tensor. */
RKNN_QUERY_OUTPUT_ATTR = 2, /* query the attribute of output tensor. */
RKNN_QUERY_PERF_DETAIL = 3, /* query the detail performance, need set
RKNN_FLAG_COLLECT_PERF_MASK when call rknn_init,
this query needs to be valid after rknn_outputs_get. */
RKNN_QUERY_PERF_RUN = 4, /* query the time of run,
this query needs to be valid after rknn_outputs_get. */
RKNN_QUERY_SDK_VERSION = 5, /* query the sdk & driver version */
RKNN_QUERY_MEM_SIZE = 6, /* query the weight & internal memory size */
RKNN_QUERY_CUSTOM_STRING = 7, /* query the custom string */
RKNN_QUERY_NATIVE_INPUT_ATTR = 8, /* query the attribute of native input tensor. */
RKNN_QUERY_NATIVE_OUTPUT_ATTR = 9, /* query the attribute of native output tensor. */
RKNN_QUERY_NATIVE_NC1HWC2_INPUT_ATTR = 8, /* query the attribute of native input tensor. */
RKNN_QUERY_NATIVE_NC1HWC2_OUTPUT_ATTR = 9, /* query the attribute of native output tensor. */
RKNN_QUERY_NATIVE_NHWC_INPUT_ATTR = 10, /* query the attribute of native input tensor. */
RKNN_QUERY_NATIVE_NHWC_OUTPUT_ATTR = 11, /* query the attribute of native output tensor. */
RKNN_QUERY_DEVICE_MEM_INFO = 12, /* query the attribute of rknn memory information. */
RKNN_QUERY_INPUT_DYNAMIC_RANGE = 13, /* query the dynamic shape range of rknn input tensor. */
RKNN_QUERY_CURRENT_INPUT_ATTR = 14, /* query the current shape of rknn input tensor, only valid for dynamic rknn model*/
RKNN_QUERY_CURRENT_OUTPUT_ATTR = 15, /* query the current shape of rknn output tensor, only valid for dynamic rknn model*/
RKNN_QUERY_CURRENT_NATIVE_INPUT_ATTR = 16, /* query the current native shape of rknn input tensor, only valid for dynamic rknn model*/
RKNN_QUERY_CURRENT_NATIVE_OUTPUT_ATTR = 17, /* query the current native shape of rknn output tensor, only valid for dynamic rknn model*/
RKNN_QUERY_CMD_MAX
} rknn_query_cmd;
/*
the tensor data type.
*/
typedef enum _rknn_tensor_type {
RKNN_TENSOR_FLOAT32 = 0, /* data type is float32. */
RKNN_TENSOR_FLOAT16, /* data type is float16. */
RKNN_TENSOR_INT8, /* data type is int8. */
RKNN_TENSOR_UINT8, /* data type is uint8. */
RKNN_TENSOR_INT16, /* data type is int16. */
RKNN_TENSOR_UINT16, /* data type is uint16. */
RKNN_TENSOR_INT32, /* data type is int32. */
RKNN_TENSOR_UINT32, /* data type is uint32. */
RKNN_TENSOR_INT64, /* data type is int64. */
RKNN_TENSOR_BOOL,
RKNN_TENSOR_INT4,
RKNN_TENSOR_BFLOAT16,
RKNN_TENSOR_TYPE_MAX
} rknn_tensor_type;
inline static const char* get_type_string(rknn_tensor_type type)
{
switch(type) {
case RKNN_TENSOR_FLOAT32: return "FP32";
case RKNN_TENSOR_FLOAT16: return "FP16";
case RKNN_TENSOR_INT8: return "INT8";
case RKNN_TENSOR_UINT8: return "UINT8";
case RKNN_TENSOR_INT16: return "INT16";
case RKNN_TENSOR_UINT16: return "UINT16";
case RKNN_TENSOR_INT32: return "INT32";
case RKNN_TENSOR_UINT32: return "UINT32";
case RKNN_TENSOR_INT64: return "INT64";
case RKNN_TENSOR_BOOL: return "BOOL";
case RKNN_TENSOR_INT4: return "INT4";
case RKNN_TENSOR_BFLOAT16: return "BF16";
default: return "UNKNOW";
}
}
/*
the quantitative type.
*/
typedef enum _rknn_tensor_qnt_type {
RKNN_TENSOR_QNT_NONE = 0, /* none. */
RKNN_TENSOR_QNT_DFP, /* dynamic fixed point. */
RKNN_TENSOR_QNT_AFFINE_ASYMMETRIC, /* asymmetric affine. */
RKNN_TENSOR_QNT_MAX
} rknn_tensor_qnt_type;
inline static const char* get_qnt_type_string(rknn_tensor_qnt_type type)
{
switch(type) {
case RKNN_TENSOR_QNT_NONE: return "NONE";
case RKNN_TENSOR_QNT_DFP: return "DFP";
case RKNN_TENSOR_QNT_AFFINE_ASYMMETRIC: return "AFFINE";
default: return "UNKNOW";
}
}
/*
the tensor data format.
*/
typedef enum _rknn_tensor_format {
RKNN_TENSOR_NCHW = 0, /* data format is NCHW. */
RKNN_TENSOR_NHWC, /* data format is NHWC. */
RKNN_TENSOR_NC1HWC2, /* data format is NC1HWC2. */
RKNN_TENSOR_UNDEFINED,
RKNN_TENSOR_FORMAT_MAX
} rknn_tensor_format;
/*
the mode of running on target NPU core.
*/
typedef enum _rknn_core_mask {
RKNN_NPU_CORE_AUTO = 0, /* default, run on NPU core randomly. */
RKNN_NPU_CORE_0 = 1, /* run on NPU core 0. */
RKNN_NPU_CORE_1 = 2, /* run on NPU core 1. */
RKNN_NPU_CORE_2 = 4, /* run on NPU core 2. */
RKNN_NPU_CORE_0_1 = RKNN_NPU_CORE_0 | RKNN_NPU_CORE_1, /* run on NPU core 0 and core 1. */
RKNN_NPU_CORE_0_1_2 = RKNN_NPU_CORE_0_1 | RKNN_NPU_CORE_2, /* run on NPU core 0 and core 1 and core 2. */
RKNN_NPU_CORE_ALL = 0xffff, /* auto choice, run on NPU cores depending on platform */
RKNN_NPU_CORE_UNDEFINED,
} rknn_core_mask;
inline static const char* get_format_string(rknn_tensor_format fmt)
{
switch(fmt) {
case RKNN_TENSOR_NCHW: return "NCHW";
case RKNN_TENSOR_NHWC: return "NHWC";
case RKNN_TENSOR_NC1HWC2: return "NC1HWC2";
case RKNN_TENSOR_UNDEFINED: return "UNDEFINED";
default: return "UNKNOW";
}
}
/*
the information for RKNN_QUERY_IN_OUT_NUM.
*/
typedef struct _rknn_input_output_num {
uint32_t n_input; /* the number of input. */
uint32_t n_output; /* the number of output. */
} rknn_input_output_num;
/*
the information for RKNN_QUERY_INPUT_ATTR / RKNN_QUERY_OUTPUT_ATTR.
*/
typedef struct _rknn_tensor_attr {
uint32_t index; /* input parameter, the index of input/output tensor,
need set before call rknn_query. */
uint32_t n_dims; /* the number of dimensions. */
uint32_t dims[RKNN_MAX_DIMS]; /* the dimensions array. */
char name[RKNN_MAX_NAME_LEN]; /* the name of tensor. */
uint32_t n_elems; /* the number of elements. */
uint32_t size; /* the bytes size of tensor. */
rknn_tensor_format fmt; /* the data format of tensor. */
rknn_tensor_type type; /* the data type of tensor. */
rknn_tensor_qnt_type qnt_type; /* the quantitative type of tensor. */
int8_t fl; /* fractional length for RKNN_TENSOR_QNT_DFP. */
int32_t zp; /* zero point for RKNN_TENSOR_QNT_AFFINE_ASYMMETRIC. */
float scale; /* scale for RKNN_TENSOR_QNT_AFFINE_ASYMMETRIC. */
uint32_t w_stride; /* the stride of tensor along the width dimention of input,
Note: it is read-only, 0 means equal to width. */
uint32_t size_with_stride; /* the bytes size of tensor with stride. */
uint8_t pass_through; /* pass through mode, for rknn_set_io_mem interface.
if TRUE, the buf data is passed directly to the input node of the rknn model
without any conversion. the following variables do not need to be set.
if FALSE, the buf data is converted into an input consistent with the model
according to the following type and fmt. so the following variables
need to be set.*/
uint32_t h_stride; /* the stride along the height dimention of input,
Note: it is write-only, if it was set to 0, h_stride = height. */
} rknn_tensor_attr;
typedef struct _rknn_input_range {
uint32_t index; /* input parameter, the index of input/output tensor,
need set before call rknn_query. */
uint32_t shape_number; /* the number of shape. */
rknn_tensor_format fmt; /* the data format of tensor. */
char name[RKNN_MAX_NAME_LEN]; /* the name of tensor. */
uint32_t dyn_range[RKNN_MAX_DYNAMIC_SHAPE_NUM][RKNN_MAX_DIMS]; /* the dynamic input dimensions range. */
uint32_t n_dims; /* the number of dimensions. */
} rknn_input_range;
/*
the information for RKNN_QUERY_PERF_DETAIL.
*/
typedef struct _rknn_perf_detail {
char* perf_data; /* the string pointer of perf detail. don't need free it by user. */
uint64_t data_len; /* the string length. */
} rknn_perf_detail;
/*
the information for RKNN_QUERY_PERF_RUN.
*/
typedef struct _rknn_perf_run {
int64_t run_duration; /* real inference time (us) */
} rknn_perf_run;
/*
the information for RKNN_QUERY_SDK_VERSION.
*/
typedef struct _rknn_sdk_version {
char api_version[256]; /* the version of rknn api. */
char drv_version[256]; /* the version of rknn driver. */
} rknn_sdk_version;
/*
the information for RKNN_QUERY_MEM_SIZE.
*/
typedef struct _rknn_mem_size {
uint32_t total_weight_size; /* the weight memory size */
uint32_t total_internal_size; /* the internal memory size, exclude inputs/outputs */
uint64_t total_dma_allocated_size; /* total dma memory allocated size */
uint32_t total_sram_size; /* total system sram size reserved for rknn */
uint32_t free_sram_size; /* free system sram size reserved for rknn */
uint32_t reserved[10]; /* reserved */
} rknn_mem_size;
/*
the information for RKNN_QUERY_CUSTOM_STRING.
*/
typedef struct _rknn_custom_string {
char string[1024]; /* the string of custom, lengths max to 1024 bytes */
} rknn_custom_string;
/*
The flags of rknn_tensor_mem.
*/
typedef enum _rknn_tensor_mem_flags {
RKNN_TENSOR_MEMORY_FLAGS_ALLOC_INSIDE = 1, /*Used to mark in rknn_destroy_mem() whether it is necessary to release the "mem" pointer itself.
If the flag RKNN_TENSOR_MEMORY_FLAGS_ALLOC_INSIDE is set, rknn_destroy_mem() will call free(mem).*/
RKNN_TENSOR_MEMORY_FLAGS_FROM_FD = 2, /*Used to mark in rknn_create_mem_from_fd() whether it is necessary to release the "mem" pointer itself.
If the flag RKNN_TENSOR_MEMORY_FLAGS_FROM_FD is set, rknn_destroy_mem() will call free(mem).*/
RKNN_TENSOR_MEMORY_FLAGS_FROM_PHYS = 3, /*Used to mark in rknn_create_mem_from_phys() whether it is necessary to release the "mem" pointer itself.
If the flag RKNN_TENSOR_MEMORY_FLAGS_FROM_PHYS is set, rknn_destroy_mem() will call free(mem).*/
RKNN_TENSOR_MEMORY_FLAGS_UNKNOWN
} rknn_tensor_mem_flags;
/*
The mode to allocate rknn memory.
*/
typedef enum _rknn_mem_alloc_flags {
RKNN_FLAG_MEMORY_FLAGS_DEFAULT = 0 << 0, /* Same with RKNN_FLAG_MEMORY_CACHEABLE */
RKNN_FLAG_MEMORY_CACHEABLE = 1 << 0, /* Create Cacheable memory. */
RKNN_FLAG_MEMORY_NON_CACHEABLE = 1 << 1, /* Create NON-Cacheable memory. */
RKNN_FLAG_MEMORY_TRY_ALLOC_SRAM = 1 << 2, /* Try to allocate memory in SRAM if possible. if SRAM is not enough, allocate rest memory in DRAM. */
} rknn_mem_alloc_flags;
/*
The mode to sync cacheable rknn memory.
*/
typedef enum _rknn_mem_sync_mode {
RKNN_MEMORY_SYNC_TO_DEVICE = 0x1, /* the mode used for consistency of device access after CPU accesses data. */
RKNN_MEMORY_SYNC_FROM_DEVICE = 0x2, /* the mode used for consistency of CPU access after device accesses data. */
RKNN_MEMORY_SYNC_BIDIRECTIONAL = RKNN_MEMORY_SYNC_TO_DEVICE | RKNN_MEMORY_SYNC_FROM_DEVICE, /* the mode used for consistency of data access
between device and CPU in both directions. */
} rknn_mem_sync_mode;
/*
the memory information of tensor.
*/
typedef struct _rknn_tensor_memory {
void* virt_addr; /* the virtual address of tensor buffer. */
uint64_t phys_addr; /* the physical address of tensor buffer. */
int32_t fd; /* the fd of tensor buffer. */
int32_t offset; /* indicates the offset of the memory. */
uint32_t size; /* the size of tensor buffer. */
uint32_t flags; /* the flags of tensor buffer, reserved */
void * priv_data; /* the private data of tensor buffer. */
} rknn_tensor_mem;
/*
the input information for rknn_input_set.
*/
typedef struct _rknn_input {
uint32_t index; /* the input index. */
void* buf; /* the input buf for index. */
uint32_t size; /* the size of input buf. */
uint8_t pass_through; /* pass through mode.
if TRUE, the buf data is passed directly to the input node of the rknn model
without any conversion. the following variables do not need to be set.
if FALSE, the buf data is converted into an input consistent with the model
according to the following type and fmt. so the following variables
need to be set.*/
rknn_tensor_type type; /* the data type of input buf. */
rknn_tensor_format fmt; /* the data format of input buf.
currently the internal input format of NPU is NCHW by default.
so entering NCHW data can avoid the format conversion in the driver. */
} rknn_input;
/*
the output information for rknn_outputs_get.
*/
typedef struct _rknn_output {
uint8_t want_float; /* want transfer output data to float */
uint8_t is_prealloc; /* whether buf is pre-allocated.
if TRUE, the following variables need to be set.
if FALSE, the following variables do not need to be set. */
uint32_t index; /* the output index. */
void* buf; /* the output buf for index.
when is_prealloc = FALSE and rknn_outputs_release called,
this buf pointer will be free and don't use it anymore. */
uint32_t size; /* the size of output buf. */
} rknn_output;
/*
the extend information for rknn_init.
*/
typedef struct _rknn_init_extend {
rknn_context ctx; /* rknn context */
int32_t real_model_offset; /* real rknn model file offset, only valid when init context with rknn file path and zero-copy model model */
uint32_t real_model_size; /* real rknn model file size, only valid when init context with rknn file path and zero-copy model model */
int32_t model_buffer_fd; /* the fd of model buffer. */
uint32_t model_buffer_flags; /* the flags of model_buffer */
uint8_t reserved[112]; /* reserved */
} rknn_init_extend;
/*
the extend information for rknn_run.
*/
typedef struct _rknn_run_extend {
uint64_t frame_id; /* output parameter, indicate current frame id of run. */
int32_t non_block; /* block flag of run, 0 is block else 1 is non block */
int32_t timeout_ms; /* timeout for block mode, in milliseconds */
int32_t fence_fd; /* fence fd from other unit */
} rknn_run_extend;
/*
the extend information for rknn_outputs_get.
*/
typedef struct _rknn_output_extend {
uint64_t frame_id; /* output parameter, indicate the frame id of outputs, corresponds to
struct rknn_run_extend.frame_id.*/
} rknn_output_extend;
/* rknn_init
initial the context and load the rknn model.
input:
rknn_context* context the pointer of context handle.
void* model if size > 0, pointer to the rknn model, if size = 0, filepath to the rknn model.
uint32_t size the size of rknn model.
uint32_t flag extend flag, see the define of RKNN_FLAG_XXX_XXX.
rknn_init_extend* extend the extend information of init.
return:
int error code.
*/
int rknn_init(rknn_context* context, void* model, uint32_t size, uint32_t flag, rknn_init_extend* extend);
/* rknn_dup_context
initial the context and load the rknn model.
input:
rknn_context* context_in the pointer of context in handle.
rknn_context* context_out the pointer of context out handle.
return:
int error code.
*/
int rknn_dup_context(rknn_context* context_in, rknn_context* context_out);
/* rknn_destroy
unload the rknn model and destroy the context.
input:
rknn_context context the handle of context.
return:
int error code.
*/
int rknn_destroy(rknn_context context);
/* rknn_query
query the information about model or others. see rknn_query_cmd.
input:
rknn_context context the handle of context.
rknn_query_cmd cmd the command of query.
void* info the buffer point of information.
uint32_t size the size of information.
return:
int error code.
*/
int rknn_query(rknn_context context, rknn_query_cmd cmd, void* info, uint32_t size);
/* rknn_inputs_set
set inputs information by input index of rknn model.
inputs information see rknn_input.
input:
rknn_context context the handle of context.
uint32_t n_inputs the number of inputs.
rknn_input inputs[] the arrays of inputs information, see rknn_input.
return:
int error code
*/
int rknn_inputs_set(rknn_context context, uint32_t n_inputs, rknn_input inputs[]);
/*
rknn_set_batch_core_num
set rknn batch core_num.
input:
rknn_context context the handle of context.
int core_num the core number.
return:
int error code.
*/
int rknn_set_batch_core_num(rknn_context context, int core_num);
/* rknn_set_core_mask
set the core mask for the model.(only supported on multi-core NPU platform)
RKNN_NPU_CORE_AUTO: auto mode, default value
RKNN_NPU_CORE_0: core 0 mode
RKNN_NPU_CORE_1: core 1 mode
RKNN_NPU_CORE_2: core 2 mode
RKNN_NPU_CORE_0_1: combine core 0/1 mode
RKNN_NPU_CORE_0_1_2: combine core 0/1/2 mode
RKNN_NPU_CORE_ALL: auto mode, select multiple npu cores to run depending on platform
input:
rknn_context context the handle of context.
rknn_core_mask core_mask the core mask.
return:
int error code.
*/
int rknn_set_core_mask(rknn_context context, rknn_core_mask core_mask);
/* rknn_run
run the model to execute inference.
input:
rknn_context context the handle of context.
rknn_run_extend* extend the extend information of run.
return:
int error code.
*/
int rknn_run(rknn_context context, rknn_run_extend* extend);
/* rknn_wait
wait the model after execute inference.
input:
rknn_context context the handle of context.
rknn_run_extend* extend the extend information of run.
return:
int error code.
*/
int rknn_wait(rknn_context context, rknn_run_extend* extend);
/* rknn_outputs_get
wait the inference to finish and get the outputs.
this function will block until inference finish.
the results will set to outputs[].
input:
rknn_context context the handle of context.
uint32_t n_outputs the number of outputs.
rknn_output outputs[] the arrays of output, see rknn_output.
rknn_output_extend* the extend information of output.
return:
int error code.
*/
int rknn_outputs_get(rknn_context context, uint32_t n_outputs, rknn_output outputs[], rknn_output_extend* extend);
/* rknn_outputs_release
release the outputs that get by rknn_outputs_get.
after called, the rknn_output[x].buf get from rknn_outputs_get will
also be free when rknn_output[x].is_prealloc = FALSE.
input:
rknn_context context the handle of context.
uint32_t n_ouputs the number of outputs.
rknn_output outputs[] the arrays of output.
return:
int error code
*/
int rknn_outputs_release(rknn_context context, uint32_t n_ouputs, rknn_output outputs[]);
/* new api for zero copy */
/* rknn_create_mem_from_phys (memory allocated outside)
initialize tensor memory from physical address.
input:
rknn_context ctx the handle of context.
uint64_t phys_addr physical address.
void *virt_addr virtual address.
uint32_t size the size of tensor buffer.
return:
rknn_tensor_mem the pointer of tensor memory information.
*/
rknn_tensor_mem* rknn_create_mem_from_phys(rknn_context ctx, uint64_t phys_addr, void *virt_addr, uint32_t size);
/* rknn_create_mem_from_fd (memory allocated outside)
initialize tensor memory from file description.
input:
rknn_context ctx the handle of context.
int32_t fd file description.
void *virt_addr virtual address.
uint32_t size the size of tensor buffer.
int32_t offset indicates the offset of the memory (virt_addr without offset).
return:
rknn_tensor_mem the pointer of tensor memory information.
*/
rknn_tensor_mem* rknn_create_mem_from_fd(rknn_context ctx, int32_t fd, void *virt_addr, uint32_t size, int32_t offset);
/* rknn_create_mem_from_mb_blk (memory allocated outside)
create tensor memory from mb_blk.
input:
rknn_context ctx the handle of context.
void *mb_blk mb_blk allocate from system api.
int32_t offset indicates the offset of the memory.
return:
rknn_tensor_mem the pointer of tensor memory information.
*/
rknn_tensor_mem* rknn_create_mem_from_mb_blk(rknn_context ctx, void *mb_blk, int32_t offset);
/* rknn_create_mem (memory allocated inside)
create tensor memory.
input:
rknn_context ctx the handle of context.
uint32_t size the size of tensor buffer.
return:
rknn_tensor_mem the pointer of tensor memory information.
*/
rknn_tensor_mem* rknn_create_mem(rknn_context ctx, uint32_t size);
/* rknn_create_mem2 (memory allocated inside)
create tensor memory.
input:
rknn_context ctx the handle of context.
uint64_t size the size of tensor buffer.
uint64_t alloc_flags memory allocation flags.
return:
rknn_tensor_mem the pointer of tensor memory information.
*/
rknn_tensor_mem* rknn_create_mem2(rknn_context ctx, uint64_t size, uint64_t alloc_flags);
/* rknn_destroy_mem (support allocate inside and outside)
destroy tensor memory.
input:
rknn_context ctx the handle of context.
rknn_tensor_mem *mem the pointer of tensor memory information.
return:
int error code
*/
int rknn_destroy_mem(rknn_context ctx, rknn_tensor_mem *mem);
/* rknn_set_weight_mem
set the weight memory.
input:
rknn_context ctx the handle of context.
rknn_tensor_mem *mem the array of tensor memory information
return:
int error code.
*/
int rknn_set_weight_mem(rknn_context ctx, rknn_tensor_mem *mem);
/* rknn_set_internal_mem
set the internal memory.
input:
rknn_context ctx the handle of context.
rknn_tensor_mem *mem the array of tensor memory information
return:
int error code.
*/
int rknn_set_internal_mem(rknn_context ctx, rknn_tensor_mem *mem);
/* rknn_set_io_mem
set the input and output tensors buffer.
input:
rknn_context ctx the handle of context.
rknn_tensor_mem *mem the array of tensor memory information.
rknn_tensor_attr *attr the attribute of input or output tensor buffer.
return:
int error code.
*/
int rknn_set_io_mem(rknn_context ctx, rknn_tensor_mem *mem, rknn_tensor_attr *attr);
/* rknn_set_input_shape(deprecated)
set the input tensor shape (only valid for dynamic shape rknn model).
input:
rknn_context ctx the handle of context.
rknn_tensor_attr *attr the attribute of input or output tensor buffer.
return:
int error code.
*/
int rknn_set_input_shape(rknn_context ctx, rknn_tensor_attr* attr);
/* rknn_set_input_shapes
set all the input tensor shapes. graph will run under current set of input shapes after rknn_set_input_shapes.(only valid for dynamic shape rknn model).
input:
rknn_context ctx the handle of context.
uint32_t n_inputs the number of inputs.
rknn_tensor_attr attr[] the attribute array of all input tensors.
return:
int error code.
*/
int rknn_set_input_shapes(rknn_context ctx, uint32_t n_inputs, rknn_tensor_attr attr[]);
/* rknn_mem_sync
sync cacheable rknn memory when both cpu and device access data.
input:
rknn_context context the handle of context.
rknn_tensor_mem *mem the pointer of tensor memory information.
rknn_mem_sync_mode mode the mode of sync cache.
return:
int error code.
*/
int rknn_mem_sync(rknn_context context, rknn_tensor_mem* mem, rknn_mem_sync_mode mode);
#ifdef __cplusplus
} //extern "C"
#endif
#endif //_RKNN_API_H

7762
include/stb_image.h Normal file
View file

File diff suppressed because it is too large Load diff

191
kernels/knn_float4_topk.cl Normal file
View file

@ -0,0 +1,191 @@
#ifndef DIM
#define DIM 128
#endif
#ifndef TOPK
#define TOPK 32
#endif
#ifndef QUERY_COUNT
#define QUERY_COUNT 1
#endif
/*
* Numero di work-item che collaborano per ogni query.
*/
#ifndef LANES_PER_QUERY
#define LANES_PER_QUERY 8
#endif
/*
* Ogni work-group contiene 8 work-item.
*
* Ogni work-item calcola un TOPK locale.
* Poi il work-item 0 fonde gli 8 TOPK locali.
*/
__kernel void knn(
__global const float *db,
__global const float *queries,
__global uint *indices,
__global float *values,
uint n)
{
uint lane = get_local_id(0);
uint q = get_group_id(0);
if (q >= QUERY_COUNT)
return;
/*
* TOPK locale per ogni work-item.
*/
float best_score[TOPK];
uint best_index[TOPK];
for (uint k = 0; k < TOPK; ++k) {
best_score[k] = -3.402823466e+38f;
best_index[k] = 0xffffffff;
}
__global const float *query =
&queries[q * DIM];
/*
* Ogni lane analizza una parte del database.
*
* Esempio con 8 lane:
*
* lane 0 -> 0, 8, 16, 24, ...
* lane 1 -> 1, 9, 17, 25, ...
* ...
* lane 7 -> 7, 15, 23, 31, ...
*/
for (uint i = lane; i < n; i += LANES_PER_QUERY) {
__global const float *v =
&db[i * DIM];
float s = 0.0f;
for (uint j = 0; j < DIM; ++j)
s += v[j] * query[j];
if (s <= best_score[TOPK - 1])
continue;
int pos = TOPK - 1;
while (pos > 0 &&
s > best_score[pos - 1]) {
--pos;
}
for (int k = TOPK - 1; k > pos; --k) {
best_score[k] =
best_score[k - 1];
best_index[k] =
best_index[k - 1];
}
best_score[pos] = s;
best_index[pos] = i;
}
/*
* TOPK locali nella local memory.
*
* 8 lane × 32 elementi.
*/
__local float local_scores[LANES_PER_QUERY][TOPK];
__local uint local_indices[LANES_PER_QUERY][TOPK];
for (uint k = 0; k < TOPK; ++k) {
local_scores[lane][k] = best_score[k];
local_indices[lane][k] = best_index[k];
}
/*
* Tutti i lane devono aver scritto
* la local memory prima della riduzione.
*/
barrier(CLK_LOCAL_MEM_FENCE);
/*
* Solo lane 0 esegue il merge finale.
*/
if (lane == 0) {
float final_score[TOPK];
uint final_index[TOPK];
for (uint k = 0; k < TOPK; ++k) {
final_score[k] =
-3.402823466e+38f;
final_index[k] =
0xffffffff;
}
/*
* Fonde gli 8 TOPK locali.
*
* Al massimo 8 × 32 = 256 candidati.
*/
for (uint l = 0;
l < LANES_PER_QUERY;
++l) {
for (uint k = 0;
k < TOPK;
++k) {
float s =
local_scores[l][k];
uint idx =
local_indices[l][k];
if (idx == 0xffffffff)
continue;
if (s <= final_score[TOPK - 1])
continue;
int pos = TOPK - 1;
while (pos > 0 &&
s > final_score[pos - 1]) {
--pos;
}
for (int p = TOPK - 1;
p > pos;
--p) {
final_score[p] =
final_score[p - 1];
final_index[p] =
final_index[p - 1];
}
final_score[pos] = s;
final_index[pos] = idx;
}
}
/*
* Risultato finale della query.
*/
for (uint k = 0; k < TOPK; ++k) {
indices[q * TOPK + k] =
final_index[k];
values[q * TOPK + k] =
final_score[k];
}
}
}

1
kernels/knn_float4_topk.cl.sha256 Normal file
View file

@ -0,0 +1 @@
6c4f502438aad3dd1600a00cb7988e0e35e56b2e8e4620d44ffcb0f3610a7226 knn_float4_topk.cl

180
kernels/knn_reference.cl Normal file
View file

@ -0,0 +1,180 @@
#define DIM 128
#define TOPK 32
/*
* Numero di work-item che collaborano per ogni query.
*/
#define LANES_PER_QUERY 8
/*
* Ogni work-group contiene 8 work-item.
*
* Ogni work-item calcola un TOPK locale.
* Poi il work-item 0 fonde gli 8 TOPK locali.
*/
__kernel void knn(
__global const float *db,
__global const float *queries,
__global uint *indices,
__global float *values,
uint n)
{
uint lane = get_local_id(0);
uint q = get_group_id(0);
if (q >= 256)
return;
/*
* TOPK locale per ogni work-item.
*/
float best_score[TOPK];
uint best_index[TOPK];
for (uint k = 0; k < TOPK; ++k) {
best_score[k] = -3.402823466e+38f;
best_index[k] = 0xffffffff;
}
__global const float *query =
&queries[q * DIM];
/*
* Ogni lane analizza una parte del database.
*
* Esempio con 8 lane:
*
* lane 0 -> 0, 8, 16, 24, ...
* lane 1 -> 1, 9, 17, 25, ...
* ...
* lane 7 -> 7, 15, 23, 31, ...
*/
for (uint i = lane; i < n; i += LANES_PER_QUERY) {
__global const float *v =
&db[i * DIM];
float s = 0.0f;
for (uint j = 0; j < DIM; ++j)
s += v[j] * query[j];
if (s <= best_score[TOPK - 1])
continue;
int pos = TOPK - 1;
while (pos > 0 &&
s > best_score[pos - 1]) {
--pos;
}
for (int k = TOPK - 1; k > pos; --k) {
best_score[k] =
best_score[k - 1];
best_index[k] =
best_index[k - 1];
}
best_score[pos] = s;
best_index[pos] = i;
}
/*
* TOPK locali nella local memory.
*
* 8 lane × 32 elementi.
*/
__local float local_scores[LANES_PER_QUERY][TOPK];
__local uint local_indices[LANES_PER_QUERY][TOPK];
for (uint k = 0; k < TOPK; ++k) {
local_scores[lane][k] = best_score[k];
local_indices[lane][k] = best_index[k];
}
/*
* Tutti i lane devono aver scritto
* la local memory prima della riduzione.
*/
barrier(CLK_LOCAL_MEM_FENCE);
/*
* Solo lane 0 esegue il merge finale.
*/
if (lane == 0) {
float final_score[TOPK];
uint final_index[TOPK];
for (uint k = 0; k < TOPK; ++k) {
final_score[k] =
-3.402823466e+38f;
final_index[k] =
0xffffffff;
}
/*
* Fonde gli 8 TOPK locali.
*
* Al massimo 8 × 32 = 256 candidati.
*/
for (uint l = 0;
l < LANES_PER_QUERY;
++l) {
for (uint k = 0;
k < TOPK;
++k) {
float s =
local_scores[l][k];
uint idx =
local_indices[l][k];
if (idx == 0xffffffff)
continue;
if (s <= final_score[TOPK - 1])
continue;
int pos = TOPK - 1;
while (pos > 0 &&
s > final_score[pos - 1]) {
--pos;
}
for (int p = TOPK - 1;
p > pos;
--p) {
final_score[p] =
final_score[p - 1];
final_index[p] =
final_index[p - 1];
}
final_score[pos] = s;
final_index[pos] = idx;
}
}
/*
* Risultato finale della query.
*/
for (uint k = 0; k < TOPK; ++k) {
indices[q * TOPK + k] =
final_index[k];
values[q * TOPK + k] =
final_score[k];
}
}
}

17
lib/README.md Normal file
View file

@ -0,0 +1,17 @@
# Runtime RKNN
Copiare in questa directory la libreria runtime:
```text
lib/librknnrt.so
```
La libreria deve essere compatibile con:
- il dispositivo Rockchip utilizzato;
- la versione del driver RKNN;
- i modelli RKNN.
Il runtime è escluso dal repository tramite `.gitignore`.
Prima della distribuzione verificare le condizioni di licenza.

25
models/README.md Normal file
View file

@ -0,0 +1,25 @@
# Modelli RKNN
Copiare in questa directory i modelli richiesti:
```text
models/SCRFD_500M_KPS_640.rknn
models/face_recognition_sface_2021dec.rknn
```
## SCRFD
Il modello SCRFD rileva i volti e produce:
- confidence score;
- bounding box;
- cinque landmark.
## SFace
Il modello SFace riceve un volto allineato a 112 x 112 pixel e produce un
embedding di 128 elementi.
I modelli sono esclusi dal repository tramite `.gitignore`.
Prima della distribuzione verificare le rispettive licenze.

48
modules/hdbscan/README.md Normal file
View file

@ -0,0 +1,48 @@
# HDBSCAN
Modulo futuro per completare la pipeline di clustering degli embedding
facciali.
## Pipeline prevista
```text
kNN graph
|
v
core distance
|
v
mutual reachability
|
v
minimum spanning tree
|
v
single linkage hierarchy
|
v
condensed tree
|
v
cluster stability
|
v
cluster assignment
```
## Output previsto
Per ogni embedding:
```text
face_index
cluster_id
membership_strength
noise_flag
```
Il risultato del clustering resterà distinto dall'identità semantica
assegnata dall'utente.
Un cluster automatico non rappresenta necessariamente una persona
definitiva e più cluster potranno essere associati alla stessa persona.

43
modules/knn-opencl/README.md Normal file
View file

@ -0,0 +1,43 @@
# kNN OpenCL
Modulo futuro per costruire il grafo dei vicini degli embedding SFace
tramite OpenCL.
## Input previsto
```text
embedding float32[N][128]
```
Gli embedding saranno già L2-normalizzati.
Di conseguenza:
```text
cosine_similarity(a,b) = dot(a,b)
```
## Output previsto
```text
indices[N][32]
scores[N][32]
```
## Obiettivo iniziale
```text
embedding float32[N][128]
|
v
brute-force dot product
|
v
TOPK=32
|
+-- neighbor index
`-- similarity score
```
La prima implementazione privilegerà la correttezza e la validazione
CPU/GPU.

29
modules/knn-opencl/reference/VALIDATION.md Normal file
View file

@ -0,0 +1,29 @@
# Validazione kNN OpenCL Mali-G610
## Parametri
- Database: 350000 embedding
- Query: 256
- Dimensione: 128
- TOPK: 32
- Chunk: 50000
- GPU: Mali-G610 r0p0
## Correttezza
- Posizioni TOPK confrontate: 8192
- Posizioni identiche CPU/OpenCL: 8192
- Exact ratio: 100.000%
## Prestazioni
- CPU: 9663.226 ms
- OpenCL: 1373.797 ms
- Speedup: 7.034x
- Throughput: 65220680 vettori/s
## Variante float4 min-replace
- Kernel: 1244.576 ms
- Throughput: 71.992 M vettori/s
- Invalid indices: 0

968
modules/knn-opencl/reference/knn_benchmark.cpp Normal file
View file

@ -0,0 +1,968 @@
#include <CL/cl.h>
#include <algorithm>
#include <chrono>
#include <cmath>
#include <cstdint>
#include <cstdlib>
#include <iomanip>
#include <iostream>
#include <random>
#include <string>
#include <vector>
static constexpr uint32_t DIM = 128;
static constexpr uint32_t TOPK = 32;
static constexpr uint32_t QUERIES = 256;
// Numero massimo di vettori elaborati da un singolo kernel launch.
static constexpr size_t CHUNK_SIZE = 50000;
static void check_cl(cl_int err, const char *what) {
if (err != CL_SUCCESS) {
std::cerr << "OpenCL error " << err << " at " << what << "\n";
std::exit(1);
}
}
static void normalize(float *v) {
double s = 0.0;
for (uint32_t i = 0; i < DIM; ++i)
s += double(v[i]) * double(v[i]);
float inv = 1.0f / std::sqrt(float(s));
for (uint32_t i = 0; i < DIM; ++i)
v[i] *= inv;
}
static std::vector<float> make_embeddings(size_t n) {
std::vector<float> x(n * DIM);
std::mt19937 rng(123456789);
std::normal_distribution<float> d(0.0f, 1.0f);
for (size_t i = 0; i < x.size(); ++i)
x[i] = d(rng);
for (size_t i = 0; i < n; ++i)
normalize(&x[i * DIM]);
return x;
}
struct Neighbor {
float score;
uint32_t index;
};
static void cpu_knn_one(
const std::vector<float>& db,
const float *query,
size_t n,
Neighbor *out)
{
std::vector<Neighbor> all(n);
for (size_t i = 0; i < n; ++i) {
const float *v = &db[i * DIM];
float s = 0.0f;
for (uint32_t j = 0; j < DIM; ++j)
s += v[j] * query[j];
all[i] = {s, static_cast<uint32_t>(i)};
}
std::nth_element(
all.begin(),
all.begin() + TOPK,
all.end(),
[](const Neighbor& a, const Neighbor& b) {
return a.score > b.score;
});
std::sort(
all.begin(),
all.begin() + TOPK,
[](const Neighbor& a, const Neighbor& b) {
return a.score > b.score;
});
for (uint32_t k = 0; k < TOPK; ++k)
out[k] = all[k];
}
static double benchmark_cpu(
const std::vector<float>& db,
const std::vector<float>& queries,
std::vector<Neighbor>& result,
size_t n)
{
result.resize(QUERIES * TOPK);
auto t0 = std::chrono::steady_clock::now();
for (uint32_t q = 0; q < QUERIES; ++q) {
cpu_knn_one(
db,
&queries[q * DIM],
n,
&result[q * TOPK]);
}
auto t1 = std::chrono::steady_clock::now();
return std::chrono::duration<double, std::milli>(t1 - t0).count();
}
static std::string cl_string(cl_device_id dev, cl_device_info info) {
size_t size = 0;
check_cl(
clGetDeviceInfo(dev, info, 0, nullptr, &size),
"clGetDeviceInfo");
std::string s(size, '\0');
check_cl(
clGetDeviceInfo(dev, info, size, s.data(), nullptr),
"clGetDeviceInfo");
while (!s.empty() && s.back() == '\0')
s.pop_back();
return s;
}
static cl_device_id find_mali() {
cl_uint np = 0;
check_cl(
clGetPlatformIDs(0, nullptr, &np),
"clGetPlatformIDs");
std::vector<cl_platform_id> platforms(np);
check_cl(
clGetPlatformIDs(np, platforms.data(), nullptr),
"clGetPlatformIDs");
for (auto p : platforms) {
cl_uint nd = 0;
if (clGetDeviceIDs(
p,
CL_DEVICE_TYPE_GPU,
0,
nullptr,
&nd) != CL_SUCCESS)
continue;
std::vector<cl_device_id> devices(nd);
check_cl(
clGetDeviceIDs(
p,
CL_DEVICE_TYPE_GPU,
nd,
devices.data(),
nullptr),
"clGetDeviceIDs");
for (auto d : devices) {
std::string name = cl_string(d, CL_DEVICE_NAME);
std::string vendor = cl_string(d, CL_DEVICE_VENDOR);
std::cout
<< "OpenCL GPU: "
<< vendor
<< " / "
<< name
<< "\n";
if (name.find("Mali-G610") != std::string::npos)
return d;
}
}
return nullptr;
}
static void merge_topk(
const std::vector<uint32_t>& chunk_indices,
const std::vector<float>& chunk_values,
size_t chunk_offset,
std::vector<Neighbor>& result)
{
/*
* Per ogni query abbiamo TOPK candidati dal chunk corrente.
*
* Il risultato globale viene mantenuto in:
*
* result[q * TOPK ... q * TOPK + TOPK)
*
* Uniamo quindi al massimo 2*TOPK elementi per query.
*/
for (uint32_t q = 0; q < QUERIES; ++q) {
std::vector<Neighbor> candidates;
candidates.reserve(TOPK * 2);
Neighbor *dst = &result[q * TOPK];
for (uint32_t k = 0; k < TOPK; ++k) {
candidates.push_back(dst[k]);
}
for (uint32_t k = 0; k < TOPK; ++k) {
uint32_t local_index =
chunk_indices[q * TOPK + k];
float score =
chunk_values[q * TOPK + k];
if (local_index == 0xffffffffu)
continue;
candidates.push_back({
score,
static_cast<uint32_t>(
chunk_offset + local_index)
});
}
std::sort(
candidates.begin(),
candidates.end(),
[](const Neighbor& a, const Neighbor& b) {
return a.score > b.score;
});
for (uint32_t k = 0; k < TOPK; ++k)
dst[k] = candidates[k];
}
}
static double benchmark_opencl(
const std::vector<float>& db,
const std::vector<float>& queries,
std::vector<Neighbor>& result,
size_t n)
{
cl_int err;
cl_device_id dev = find_mali();
if (!dev) {
std::cerr << "Mali-G610 not found\n";
std::exit(1);
}
cl_context ctx =
clCreateContext(
nullptr,
1,
&dev,
nullptr,
nullptr,
&err);
check_cl(err, "clCreateContext");
cl_command_queue queue =
clCreateCommandQueue(ctx, dev, 0, &err);
check_cl(err, "clCreateCommandQueue");
const char *source = R"CLC(
#define DIM 128
#define TOPK 32
/*
* Numero di work-item che collaborano per ogni query.
*/
#define LANES_PER_QUERY 8
/*
* Ogni work-group contiene 8 work-item.
*
* Ogni work-item calcola un TOPK locale.
* Poi il work-item 0 fonde gli 8 TOPK locali.
*/
__kernel void knn(
__global const float *db,
__global const float *queries,
__global uint *indices,
__global float *values,
uint n)
{
uint lane = get_local_id(0);
uint q = get_group_id(0);
if (q >= 256)
return;
/*
* TOPK locale per ogni work-item.
*/
float best_score[TOPK];
uint best_index[TOPK];
for (uint k = 0; k < TOPK; ++k) {
best_score[k] = -3.402823466e+38f;
best_index[k] = 0xffffffff;
}
__global const float *query =
&queries[q * DIM];
/*
* Ogni lane analizza una parte del database.
*
* Esempio con 8 lane:
*
* lane 0 -> 0, 8, 16, 24, ...
* lane 1 -> 1, 9, 17, 25, ...
* ...
* lane 7 -> 7, 15, 23, 31, ...
*/
for (uint i = lane; i < n; i += LANES_PER_QUERY) {
__global const float *v =
&db[i * DIM];
float s = 0.0f;
for (uint j = 0; j < DIM; ++j)
s += v[j] * query[j];
if (s <= best_score[TOPK - 1])
continue;
int pos = TOPK - 1;
while (pos > 0 &&
s > best_score[pos - 1]) {
--pos;
}
for (int k = TOPK - 1; k > pos; --k) {
best_score[k] =
best_score[k - 1];
best_index[k] =
best_index[k - 1];
}
best_score[pos] = s;
best_index[pos] = i;
}
/*
* TOPK locali nella local memory.
*
* 8 lane × 32 elementi.
*/
__local float local_scores[LANES_PER_QUERY][TOPK];
__local uint local_indices[LANES_PER_QUERY][TOPK];
for (uint k = 0; k < TOPK; ++k) {
local_scores[lane][k] = best_score[k];
local_indices[lane][k] = best_index[k];
}
/*
* Tutti i lane devono aver scritto
* la local memory prima della riduzione.
*/
barrier(CLK_LOCAL_MEM_FENCE);
/*
* Solo lane 0 esegue il merge finale.
*/
if (lane == 0) {
float final_score[TOPK];
uint final_index[TOPK];
for (uint k = 0; k < TOPK; ++k) {
final_score[k] =
-3.402823466e+38f;
final_index[k] =
0xffffffff;
}
/*
* Fonde gli 8 TOPK locali.
*
* Al massimo 8 × 32 = 256 candidati.
*/
for (uint l = 0;
l < LANES_PER_QUERY;
++l) {
for (uint k = 0;
k < TOPK;
++k) {
float s =
local_scores[l][k];
uint idx =
local_indices[l][k];
if (idx == 0xffffffff)
continue;
if (s <= final_score[TOPK - 1])
continue;
int pos = TOPK - 1;
while (pos > 0 &&
s > final_score[pos - 1]) {
--pos;
}
for (int p = TOPK - 1;
p > pos;
--p) {
final_score[p] =
final_score[p - 1];
final_index[p] =
final_index[p - 1];
}
final_score[pos] = s;
final_index[pos] = idx;
}
}
/*
* Risultato finale della query.
*/
for (uint k = 0; k < TOPK; ++k) {
indices[q * TOPK + k] =
final_index[k];
values[q * TOPK + k] =
final_score[k];
}
}
}
)CLC";
cl_program program =
clCreateProgramWithSource(
ctx,
1,
&source,
nullptr,
&err);
check_cl(err, "clCreateProgramWithSource");
err = clBuildProgram(
program,
1,
&dev,
"-cl-fast-relaxed-math",
nullptr,
nullptr);
if (err != CL_SUCCESS) {
size_t log_size = 0;
clGetProgramBuildInfo(
program,
dev,
CL_PROGRAM_BUILD_LOG,
0,
nullptr,
&log_size);
std::string log(log_size, '\0');
clGetProgramBuildInfo(
program,
dev,
CL_PROGRAM_BUILD_LOG,
log_size,
log.data(),
nullptr);
std::cerr << log << "\n";
check_cl(err, "clBuildProgram");
}
cl_kernel kernel =
clCreateKernel(program, "knn", &err);
check_cl(err, "clCreateKernel");
/*
* Buffer GPU riutilizzato per ogni chunk.
*/
const size_t chunk_capacity =
std::min(CHUNK_SIZE, n);
cl_mem db_buf =
clCreateBuffer(
ctx,
CL_MEM_READ_ONLY,
chunk_capacity * DIM * sizeof(float),
nullptr,
&err);
check_cl(err, "db buffer");
cl_mem query_buf =
clCreateBuffer(
ctx,
CL_MEM_READ_ONLY | CL_MEM_COPY_HOST_PTR,
queries.size() * sizeof(float),
const_cast<float*>(queries.data()),
&err);
check_cl(err, "query buffer");
std::vector<uint32_t> indices(
QUERIES * TOPK);
std::vector<float> values(
QUERIES * TOPK);
cl_mem index_buf =
clCreateBuffer(
ctx,
CL_MEM_WRITE_ONLY,
indices.size() * sizeof(uint32_t),
nullptr,
&err);
check_cl(err, "index buffer");
cl_mem value_buf =
clCreateBuffer(
ctx,
CL_MEM_WRITE_ONLY,
values.size() * sizeof(float),
nullptr,
&err);
check_cl(err, "value buffer");
check_cl(
clSetKernelArg(
kernel,
0,
sizeof(db_buf),
&db_buf),
"arg0");
check_cl(
clSetKernelArg(
kernel,
1,
sizeof(query_buf),
&query_buf),
"arg1");
check_cl(
clSetKernelArg(
kernel,
2,
sizeof(index_buf),
&index_buf),
"arg2");
check_cl(
clSetKernelArg(
kernel,
3,
sizeof(value_buf),
&value_buf),
"arg3");
/*
* Risultato globale.
*
* Inizializziamo gli score al minimo possibile,
* così il primo chunk può essere mergiato
* senza una gestione speciale.
*/
result.resize(QUERIES * TOPK);
for (size_t i = 0; i < result.size(); ++i) {
result[i] = {
-3.402823466e+38f,
0xffffffffu
};
}
size_t num_chunks =
(n + CHUNK_SIZE - 1) / CHUNK_SIZE;
std::cout
<< "GPU CHUNK_SIZE : "
<< CHUNK_SIZE
<< "\n";
std::cout
<< "GPU CHUNKS : "
<< num_chunks
<< "\n";
size_t global = QUERIES * 8;
size_t local = 8;
double write_ms = 0.0;
double kernel_ms = 0.0;
double read_ms = 0.0;
auto t0 =
std::chrono::steady_clock::now();
for (size_t chunk_offset = 0;
chunk_offset < n;
chunk_offset += CHUNK_SIZE) {
size_t chunk_n =
std::min(
CHUNK_SIZE,
n - chunk_offset);
const float *chunk_ptr =
&db[chunk_offset * DIM];
/*
* Copia del chunk nel buffer GPU.
*
* La coda è in-order, quindi il write,
* kernel e read vengono eseguiti nell'ordine
* corretto.
*/
auto tw0 = std::chrono::steady_clock::now();
check_cl(
clEnqueueWriteBuffer(
queue,
db_buf,
CL_TRUE,
0,
chunk_n * DIM * sizeof(float),
chunk_ptr,
0,
nullptr,
nullptr),
"write db chunk");
auto tw1 = std::chrono::steady_clock::now();
write_ms +=
std::chrono::duration<double, std::milli>(
tw1 - tw0).count();
uint32_t nn =
static_cast<uint32_t>(chunk_n);
check_cl(
clSetKernelArg(
kernel,
4,
sizeof(nn),
&nn),
"arg4");
auto tk0 = std::chrono::steady_clock::now();
check_cl(
clEnqueueNDRangeKernel(
queue,
kernel,
1,
nullptr,
&global,
&local,
0,
nullptr,
nullptr),
"clEnqueueNDRangeKernel");
check_cl(
clFinish(queue),
"clFinish after kernel");
auto tk1 = std::chrono::steady_clock::now();
kernel_ms +=
std::chrono::duration<double, std::milli>(
tk1 - tk0).count();
auto tr0 = std::chrono::steady_clock::now();
check_cl(
clEnqueueReadBuffer(
queue,
index_buf,
CL_TRUE,
0,
indices.size() * sizeof(uint32_t),
indices.data(),
0,
nullptr,
nullptr),
"read indices");
check_cl(
clEnqueueReadBuffer(
queue,
value_buf,
CL_TRUE,
0,
values.size() * sizeof(float),
values.data(),
0,
nullptr,
nullptr),
"read values");
auto tr1 = std::chrono::steady_clock::now();
read_ms +=
std::chrono::duration<double, std::milli>(
tr1 - tr0).count();
/*
* Gli indici prodotti dal kernel sono locali
* al chunk. merge_topk() aggiunge chunk_offset
* per ottenere gli indici globali.
*/
merge_topk(
indices,
values,
chunk_offset,
result);
}
auto t1 =
std::chrono::steady_clock::now();
clReleaseMemObject(value_buf);
clReleaseMemObject(index_buf);
clReleaseMemObject(query_buf);
clReleaseMemObject(db_buf);
clReleaseKernel(kernel);
clReleaseProgram(program);
clReleaseCommandQueue(queue);
clReleaseContext(ctx);
std::cout
<< "GPU WRITE TOTAL : "
<< write_ms
<< " ms\n";
std::cout
<< "GPU KERNEL TOTAL: "
<< kernel_ms
<< " ms\n";
std::cout
<< "GPU READ TOTAL : "
<< read_ms
<< " ms\n";
return std::chrono::duration<double, std::milli>(
t1 - t0).count();
}
int main(int argc, char **argv) {
size_t n = 100000;
if (argc > 1)
n = std::strtoull(
argv[1],
nullptr,
10);
std::cout
<< "=== BATCH KNN BENCHMARK ===\n";
std::cout
<< "DATABASE : "
<< n
<< "\n";
std::cout
<< "QUERIES : "
<< QUERIES
<< "\n";
std::cout
<< "DIM : "
<< DIM
<< "\n";
std::cout
<< "TOPK : "
<< TOPK
<< "\n";
std::cout
<< "SEED : 123456789\n\n";
std::cout
<< "Generating database...\n";
auto db =
make_embeddings(n);
std::cout
<< "Generating queries...\n";
auto queries =
make_embeddings(QUERIES);
std::vector<Neighbor> cpu;
std::vector<Neighbor> gpu;
std::cout
<< "\n=== CPU ===\n";
double cpu_ms =
benchmark_cpu(
db,
queries,
cpu,
n);
std::cout
<< std::fixed
<< std::setprecision(3);
std::cout
<< "time_ms = "
<< cpu_ms
<< "\n";
std::cout
<< "q0_top1 = "
<< cpu[0].index
<< "\n";
std::cout
<< "\n=== OPENCL MALI ===\n";
double gpu_ms =
benchmark_opencl(
db,
queries,
gpu,
n);
std::cout
<< "time_ms = "
<< gpu_ms
<< "\n";
std::cout
<< "q0_top1 = "
<< gpu[0].index
<< "\n";
size_t total =
QUERIES * TOPK;
size_t same = 0;
for (size_t i = 0;
i < total;
++i) {
if (cpu[i].index == gpu[i].index)
++same;
}
std::cout
<< "exact_topk_positions = "
<< same
<< "/"
<< total
<< "\n";
std::cout
<< "exact_ratio = "
<< (double(same) /
double(total)) * 100.0
<< "%\n";
for (size_t i = 0; i < total; ++i) {
if (cpu[i].index != gpu[i].index) {
size_t q = i / TOPK;
size_t k = i % TOPK;
std::cout
<< "MISMATCH q=" << q
<< " k=" << k
<< " CPU index=" << cpu[i].index
<< " score=" << cpu[i].score
<< " GPU index=" << gpu[i].index
<< " score=" << gpu[i].score
<< "\n";
}
}
std::cout
<< "\n=== SPEEDUP ===\n";
std::cout
<< "CPU/OpenCL = "
<< cpu_ms / gpu_ms
<< "x\n";
double pairs =
double(n) *
double(QUERIES);
double billion =
pairs *
double(DIM) /
1e9;
std::cout
<< "\n=== WORK ===\n";
std::cout
<< "similarity_ops = "
<< billion
<< " billion dot-product elements\n";
std::cout
<< "\n=== THROUGHPUT ===\n";
std::cout
<< "database_vectors_per_second = "
<< (double(n) *
double(QUERIES))
/ (gpu_ms / 1000.0)
<< "\n";
return 0;
}

972
modules/knn-opencl/reference/knn_float4_minreplace_validate_benchmark.cpp Normal file
View file

@ -0,0 +1,972 @@
#define CL_TARGET_OPENCL_VERSION 300
#include <CL/cl.h>
#include <algorithm>
#include <chrono>
#include <cmath>
#include <cstdlib>
#include <iomanip>
#include <iostream>
#include <random>
#include <vector>
static constexpr int DIM = 128;
static constexpr int DIM4 = 32;
static constexpr int TOPK = 32;
static constexpr int QUERIES = 32;
static constexpr int CHUNK_SIZE = 5000;
static constexpr int LANES = 8;
static constexpr int TOTAL_DB = 12000;
static void check(cl_int err, const char* what) {
if (err != CL_SUCCESS) {
std::cerr << what << " failed: " << err << "\n";
std::exit(1);
}
}
static const char* kernel_source = R"CLC(
#define DIM4 32
#define TOPK 32
#define LANES 8
#define QUERIES 32
inline float dot_f4(
__global const float4* db,
__global const float4* queries,
uint q,
uint db_index)
{
uint db_base = db_index * DIM4;
uint q_base = q * DIM4;
float4 acc0 = (float4)(0.0f);
float4 acc1 = (float4)(0.0f);
float4 acc2 = (float4)(0.0f);
float4 acc3 = (float4)(0.0f);
for (uint d = 0; d < DIM4; d += 4) {
acc0 += db[db_base + d + 0] * queries[q_base + d + 0];
acc1 += db[db_base + d + 1] * queries[q_base + d + 1];
acc2 += db[db_base + d + 2] * queries[q_base + d + 2];
acc3 += db[db_base + d + 3] * queries[q_base + d + 3];
}
float4 s = acc0 + acc1 + acc2 + acc3;
return s.x + s.y + s.z + s.w;
}
__kernel void knn_minreplace(
__global const float4* db,
__global const float4* queries,
uint db_count,
__global float* local_scores_out,
__global uint* local_indices_out)
{
uint gid = get_global_id(0);
uint lane = gid % LANES;
uint q = gid / LANES;
if (q >= QUERIES)
return;
float best[TOPK];
uint best_idx[TOPK];
for (uint k = 0; k < TOPK; ++k) {
best[k] = -INFINITY;
best_idx[k] = 0;
}
float min_score = -INFINITY;
uint min_pos = 0;
for (uint i = lane; i < db_count; i += LANES) {
float score = dot_f4(db, queries, q, i);
if (score > min_score) {
best[min_pos] = score;
best_idx[min_pos] = i;
min_score = best[0];
min_pos = 0;
for (uint k = 1; k < TOPK; ++k) {
if (best[k] < min_score) {
min_score = best[k];
min_pos = k;
}
}
}
}
uint base = (q * LANES + lane) * TOPK;
for (uint k = 0; k < TOPK; ++k) {
local_scores_out[base + k] = best[k];
local_indices_out[base + k] = best_idx[k];
}
}
__kernel void merge_topk(
__global const float* local_scores,
__global const uint* local_indices,
__global float* final_scores,
__global uint* final_indices)
{
uint q = get_global_id(0);
if (q >= QUERIES)
return;
float best[TOPK];
uint best_idx[TOPK];
for (uint k = 0; k < TOPK; ++k) {
best[k] = -INFINITY;
best_idx[k] = 0;
}
float min_score = -INFINITY;
uint min_pos = 0;
for (uint lane = 0; lane < LANES; ++lane) {
uint base = (q * LANES + lane) * TOPK;
for (uint k = 0; k < TOPK; ++k) {
float score = local_scores[base + k];
if (score > min_score) {
best[min_pos] = score;
best_idx[min_pos] = local_indices[base + k];
min_score = best[0];
min_pos = 0;
for (uint j = 1; j < TOPK; ++j) {
if (best[j] < min_score) {
min_score = best[j];
min_pos = j;
}
}
}
}
}
// Sort final TOPK descending, to match the semantic output
// of the original benchmark.
for (uint i = 0; i < TOPK; ++i) {
uint max_pos = i;
for (uint j = i + 1; j < TOPK; ++j) {
if (best[j] > best[max_pos])
max_pos = j;
}
if (max_pos != i) {
float ts = best[i];
best[i] = best[max_pos];
best[max_pos] = ts;
uint ti = best_idx[i];
best_idx[i] = best_idx[max_pos];
best_idx[max_pos] = ti;
}
}
uint out_base = q * TOPK;
for (uint k = 0; k < TOPK; ++k) {
final_scores[out_base + k] = best[k];
final_indices[out_base + k] = best_idx[k];
}
}
)CLC";
int main() {
cl_int err;
cl_uint num_platforms = 0;
check(
clGetPlatformIDs(0, nullptr, &num_platforms),
"clGetPlatformIDs");
std::vector<cl_platform_id> platforms(num_platforms);
check(
clGetPlatformIDs(
num_platforms,
platforms.data(),
nullptr),
"clGetPlatformIDs(list)");
cl_platform_id platform = platforms[0];
cl_uint num_devices = 0;
check(
clGetDeviceIDs(
platform,
CL_DEVICE_TYPE_GPU,
0,
nullptr,
&num_devices),
"clGetDeviceIDs(count)");
std::vector<cl_device_id> devices(num_devices);
check(
clGetDeviceIDs(
platform,
CL_DEVICE_TYPE_GPU,
num_devices,
devices.data(),
nullptr),
"clGetDeviceIDs(list)");
cl_device_id device = devices[0];
char device_name[256] = {};
clGetDeviceInfo(
device,
CL_DEVICE_NAME,
sizeof(device_name),
device_name,
nullptr);
std::cout << "=== FULL KNN FLOAT4 + MIN-REPLACE ===\n";
std::cout << "Device : " << device_name << "\n";
std::cout << "DIM : " << DIM << "\n";
std::cout << "TOPK : " << TOPK << "\n";
std::cout << "QUERIES : " << QUERIES << "\n";
std::cout << "TOTAL_DB : " << TOTAL_DB << "\n";
std::cout << "CHUNK_SIZE : " << CHUNK_SIZE << "\n";
std::cout << "LANES : " << LANES << "\n\n";
cl_context context =
clCreateContext(
nullptr,
1,
&device,
nullptr,
nullptr,
&err);
check(err, "clCreateContext");
cl_command_queue queue =
clCreateCommandQueue(
context,
device,
CL_QUEUE_PROFILING_ENABLE,
&err);
check(err, "clCreateCommandQueue");
const size_t db_bytes =
static_cast<size_t>(CHUNK_SIZE) *
DIM4 *
4 *
sizeof(float);
const size_t query_bytes =
static_cast<size_t>(QUERIES) *
DIM4 *
4 *
sizeof(float);
const size_t lane_candidates =
static_cast<size_t>(QUERIES) *
LANES *
TOPK;
const size_t candidate_bytes =
lane_candidates * sizeof(float);
const size_t index_bytes =
lane_candidates * sizeof(cl_uint);
const size_t final_count =
static_cast<size_t>(QUERIES) * TOPK;
const size_t final_score_bytes =
final_count * sizeof(float);
const size_t final_index_bytes =
final_count * sizeof(cl_uint);
cl_mem db_buf =
clCreateBuffer(
context,
CL_MEM_READ_ONLY,
db_bytes,
nullptr,
&err);
check(err, "clCreateBuffer(db)");
cl_mem query_buf =
clCreateBuffer(
context,
CL_MEM_READ_ONLY,
query_bytes,
nullptr,
&err);
check(err, "clCreateBuffer(query)");
cl_mem local_scores_buf =
clCreateBuffer(
context,
CL_MEM_READ_WRITE,
candidate_bytes,
nullptr,
&err);
check(err, "clCreateBuffer(local_scores)");
cl_mem local_indices_buf =
clCreateBuffer(
context,
CL_MEM_READ_WRITE,
index_bytes,
nullptr,
&err);
check(err, "clCreateBuffer(local_indices)");
cl_mem final_scores_buf =
clCreateBuffer(
context,
CL_MEM_READ_WRITE,
final_score_bytes,
nullptr,
&err);
check(err, "clCreateBuffer(final_scores)");
cl_mem final_indices_buf =
clCreateBuffer(
context,
CL_MEM_READ_WRITE,
final_index_bytes,
nullptr,
&err);
check(err, "clCreateBuffer(final_indices)");
cl_program program =
clCreateProgramWithSource(
context,
1,
&kernel_source,
nullptr,
&err);
check(err, "clCreateProgramWithSource");
err = clBuildProgram(
program,
1,
&device,
"-cl-fast-relaxed-math",
nullptr,
nullptr);
if (err != CL_SUCCESS) {
size_t log_size = 0;
clGetProgramBuildInfo(
program,
device,
CL_PROGRAM_BUILD_LOG,
0,
nullptr,
&log_size);
std::vector<char> log(log_size + 1);
clGetProgramBuildInfo(
program,
device,
CL_PROGRAM_BUILD_LOG,
log_size,
log.data(),
nullptr);
std::cerr << log.data() << "\n";
check(err, "clBuildProgram");
}
cl_kernel knn =
clCreateKernel(
program,
"knn_minreplace",
&err);
check(err, "clCreateKernel(knn_minreplace)");
cl_kernel merge =
clCreateKernel(
program,
"merge_topk",
&err);
check(err, "clCreateKernel(merge_topk)");
std::vector<float> db(
static_cast<size_t>(TOTAL_DB) * DIM);
std::vector<float> queries(
static_cast<size_t>(QUERIES) * DIM);
std::mt19937 rng_db(123456789);
std::mt19937 rng_q(987654321);
std::normal_distribution<float> normal(0.0f, 1.0f);
for (int i = 0; i < TOTAL_DB; ++i) {
float norm = 0.0f;
for (int d = 0; d < DIM; ++d) {
float x = normal(rng_db);
db[
static_cast<size_t>(i) * DIM + d
] = x;
norm += x * x;
}
norm = 1.0f / std::sqrt(norm);
for (int d = 0; d < DIM; ++d)
db[
static_cast<size_t>(i) * DIM + d
] *= norm;
}
for (int q = 0; q < QUERIES; ++q) {
float norm = 0.0f;
for (int d = 0; d < DIM; ++d) {
float x = normal(rng_q);
queries[
static_cast<size_t>(q) * DIM + d
] = x;
norm += x * x;
}
norm = 1.0f / std::sqrt(norm);
for (int d = 0; d < DIM; ++d)
queries[
static_cast<size_t>(q) * DIM + d
] *= norm;
}
std::vector<float> final_scores(
final_count);
std::vector<cl_uint> final_indices(
final_count);
/*
* TOP-K globale persistente fra tutti i chunk.
*
* final_scores/final_indices contengono il risultato
* del chunk corrente.
*
* global_scores/global_indices contengono invece il
* TOP-K complessivo elaborato fino a quel momento.
*/
std::vector<float> global_scores(
final_count,
-INFINITY);
std::vector<cl_uint> global_indices(
final_count,
0);
double total_knn_ms = 0.0;
double total_merge_ms = 0.0;
double total_global_merge_ms = 0.0;
const int chunks =
(TOTAL_DB + CHUNK_SIZE - 1) /
CHUNK_SIZE;
for (int chunk = 0; chunk < chunks; ++chunk) {
const int begin =
chunk * CHUNK_SIZE;
const int count =
std::min(
CHUNK_SIZE,
TOTAL_DB - begin);
const size_t db_upload_bytes =
static_cast<size_t>(count) *
DIM4 *
4 *
sizeof(float);
check(
clEnqueueWriteBuffer(
queue,
db_buf,
CL_TRUE,
0,
db_upload_bytes,
db.data() +
static_cast<size_t>(begin) * DIM,
0,
nullptr,
nullptr),
"write db");
check(
clEnqueueWriteBuffer(
queue,
query_buf,
CL_TRUE,
0,
query_bytes,
queries.data(),
0,
nullptr,
nullptr),
"write queries");
check(
clSetKernelArg(
knn,
0,
sizeof(cl_mem),
&db_buf),
"arg 0");
check(
clSetKernelArg(
knn,
1,
sizeof(cl_mem),
&query_buf),
"arg 1");
cl_uint db_count = count;
check(
clSetKernelArg(
knn,
2,
sizeof(cl_uint),
&db_count),
"arg 2");
check(
clSetKernelArg(
knn,
3,
sizeof(cl_mem),
&local_scores_buf),
"arg 3");
check(
clSetKernelArg(
knn,
4,
sizeof(cl_mem),
&local_indices_buf),
"arg 4");
size_t global =
static_cast<size_t>(QUERIES) *
LANES;
size_t local = LANES;
auto t0 =
std::chrono::steady_clock::now();
check(
clEnqueueNDRangeKernel(
queue,
knn,
1,
nullptr,
&global,
&local,
0,
nullptr,
nullptr),
"enqueue knn");
check(clFinish(queue), "finish knn");
auto t1 =
std::chrono::steady_clock::now();
total_knn_ms +=
std::chrono::duration<double, std::milli>(
t1 - t0).count();
check(
clSetKernelArg(
merge,
0,
sizeof(cl_mem),
&local_scores_buf),
"merge arg 0");
check(
clSetKernelArg(
merge,
1,
sizeof(cl_mem),
&local_indices_buf),
"merge arg 1");
check(
clSetKernelArg(
merge,
2,
sizeof(cl_mem),
&final_scores_buf),
"merge arg 2");
check(
clSetKernelArg(
merge,
3,
sizeof(cl_mem),
&final_indices_buf),
"merge arg 3");
size_t merge_global = QUERIES;
auto tm0 =
std::chrono::steady_clock::now();
check(
clEnqueueNDRangeKernel(
queue,
merge,
1,
nullptr,
&merge_global,
nullptr,
0,
nullptr,
nullptr),
"enqueue merge");
check(clFinish(queue), "finish merge");
auto tm1 =
std::chrono::steady_clock::now();
total_merge_ms +=
std::chrono::duration<double, std::milli>(
tm1 - tm0).count();
check(
clEnqueueReadBuffer(
queue,
final_scores_buf,
CL_TRUE,
0,
final_score_bytes,
final_scores.data(),
0,
nullptr,
nullptr),
"read final scores");
check(
clEnqueueReadBuffer(
queue,
final_indices_buf,
CL_TRUE,
0,
final_index_bytes,
final_indices.data(),
0,
nullptr,
nullptr),
"read final indices");
/*
* Merge CPU fra:
*
* TOP-K globale accumulato finora
* TOP-K prodotto dal chunk corrente
*
* Gli indici restituiti dalla GPU sono locali al
* chunk. Li convertiamo quindi in indici globali
* aggiungendo begin.
*/
auto global_merge_start =
std::chrono::steady_clock::now();
for (int q = 0; q < QUERIES; ++q) {
const size_t base =
static_cast<size_t>(q) * TOPK;
for (int k = 0; k < TOPK; ++k) {
const float candidate_score =
final_scores[base + k];
const cl_uint candidate_index =
static_cast<cl_uint>(begin) +
final_indices[base + k];
/*
* Trova il risultato peggiore nel TOP-K
* globale corrente.
*/
size_t worst_pos = base;
float worst_score =
global_scores[base];
for (int j = 1; j < TOPK; ++j) {
const size_t pos =
base +
static_cast<size_t>(j);
if (global_scores[pos] <
worst_score)
{
worst_score =
global_scores[pos];
worst_pos = pos;
}
}
/*
* Sostituzione del minimo.
*/
if (candidate_score > worst_score) {
global_scores[worst_pos] =
candidate_score;
global_indices[worst_pos] =
candidate_index;
}
}
}
auto global_merge_end =
std::chrono::steady_clock::now();
total_global_merge_ms +=
std::chrono::duration<
double,
std::milli>(
global_merge_end -
global_merge_start).count();
}
/*
* Ordina il TOP-K globale di ogni query in ordine
* decrescente di similarita.
*/
for (int q = 0; q < QUERIES; ++q) {
const size_t base =
static_cast<size_t>(q) * TOPK;
for (int i = 0; i < TOPK; ++i) {
int max_pos = i;
for (int j = i + 1;
j < TOPK;
++j)
{
if (global_scores[
base +
static_cast<size_t>(j)
] >
global_scores[
base +
static_cast<size_t>(max_pos)
])
{
max_pos = j;
}
}
if (max_pos != i) {
std::swap(
global_scores[
base +
static_cast<size_t>(i)
],
global_scores[
base +
static_cast<size_t>(max_pos)
]);
std::swap(
global_indices[
base +
static_cast<size_t>(i)
],
global_indices[
base +
static_cast<size_t>(max_pos)
]);
}
}
}
/*
* Verifica intervallo indici globali.
*/
size_t invalid_indices = 0;
for (cl_uint index : global_indices) {
if (index >=
static_cast<cl_uint>(TOTAL_DB))
{
++invalid_indices;
}
}
double checksum_scores = 0.0;
double checksum_indices = 0.0;
for (size_t i = 0; i < final_count; ++i) {
checksum_scores += global_scores[i];
checksum_indices += global_indices[i];
}
double vectors =
static_cast<double>(TOTAL_DB) *
QUERIES;
double vector_rate =
vectors /
total_knn_ms /
1000.0;
double gflops =
vector_rate *
DIM *
2.0 /
1000.0;
double db_traffic =
vector_rate *
DIM *
sizeof(float) /
1000.0;
std::cout << "\n=== RESULT ===\n";
std::cout << "KNN kernel time : "
<< std::fixed
<< std::setprecision(3)
<< total_knn_ms
<< " ms\n";
std::cout << "Merge time : "
<< std::fixed
<< std::setprecision(3)
<< total_merge_ms
<< " ms\n";
std::cout << "Global CPU merge: "
<< std::fixed
<< std::setprecision(3)
<< total_global_merge_ms
<< " ms\n";
std::cout << "Total GPU time : "
<< std::fixed
<< std::setprecision(3)
<< total_knn_ms + total_merge_ms
<< " ms\n";
std::cout << "Search core time: "
<< std::fixed
<< std::setprecision(3)
<< total_knn_ms +
total_merge_ms +
total_global_merge_ms
<< " ms\n";
std::cout << "Invalid indices : "
<< invalid_indices
<< "\n";
std::cout << "Vector rate : "
<< std::fixed
<< std::setprecision(3)
<< vector_rate
<< " M vector/s\n";
std::cout << "Dot rate : "
<< std::fixed
<< std::setprecision(3)
<< gflops
<< " GFLOP/s\n";
std::cout << "DB traffic : "
<< std::fixed
<< std::setprecision(3)
<< db_traffic
<< " GB/s effective\n";
std::cout << "Score checksum : "
<< std::fixed
<< std::setprecision(3)
<< checksum_scores
<< "\n";
std::cout << "Index checksum : "
<< std::fixed
<< std::setprecision(3)
<< checksum_indices
<< "\n";
clReleaseKernel(knn);
clReleaseKernel(merge);
clReleaseProgram(program);
clReleaseMemObject(db_buf);
clReleaseMemObject(query_buf);
clReleaseMemObject(local_scores_buf);
clReleaseMemObject(local_indices_buf);
clReleaseMemObject(final_scores_buf);
clReleaseMemObject(final_indices_buf);
clReleaseCommandQueue(queue);
clReleaseContext(context);
return 0;
}

23
modules/knn-opencl/reference/validation-minreplace-350k.txt Normal file
View file

@ -0,0 +1,23 @@
arm_release_ver: g24p0-00eac0, rk_so_ver: 10
=== FULL KNN FLOAT4 + MIN-REPLACE ===
Device : Mali-G610 r0p0
DIM : 128
TOPK : 32
QUERIES : 256
TOTAL_DB : 350000
CHUNK_SIZE : 50000
LANES : 8
=== RESULT ===
KNN kernel time : 1244.576 ms
Merge time : 8.022 ms
Global CPU merge: 14.039 ms
Total GPU time : 1252.599 ms
Search core time: 1266.638 ms
Invalid indices : 0
Vector rate : 71.992 M vector/s
Dot rate : 18.430 GFLOP/s
DB traffic : 36.860 GB/s effective
Score checksum : 2804.191
Index checksum : 1425305049.000

35
modules/knn-opencl/reference/validation-opencl-350k.txt Normal file
View file

@ -0,0 +1,35 @@
=== BATCH KNN BENCHMARK ===
DATABASE : 350000
QUERIES : 256
DIM : 128
TOPK : 32
SEED : 123456789
Generating database...
Generating queries...
=== CPU ===
time_ms = 9663.226
q0_top1 = 0
=== OPENCL MALI ===
arm_release_ver: g24p0-00eac0, rk_so_ver: 10
OpenCL GPU: ARM / Mali-G610 r0p0
GPU CHUNK_SIZE : 50000
GPU CHUNKS : 7
GPU WRITE TOTAL : 48.242 ms
GPU KERNEL TOTAL: 1318.590 ms
GPU READ TOTAL : 0.712 ms
time_ms = 1373.797
q0_top1 = 0
exact_topk_positions = 8192/8192
exact_ratio = 100.000%
=== SPEEDUP ===
CPU/OpenCL = 7.034x
=== WORK ===
similarity_ops = 11.469 billion dot-product elements
=== THROUGHPUT ===
database_vectors_per_second = 65220680.078

29
modules/mst/README.md Normal file
View file

@ -0,0 +1,29 @@
# Minimum Spanning Tree
Modulo futuro per costruire il minimum spanning tree del grafo di mutual
reachability.
## Input previsto
Un grafo pesato:
```text
source
destination
mutual_reachability_weight
```
## Output previsto
Per un dataset con `N` vertici connessi, l'MST conterrà:
```text
N - 1 archi
```
La prima implementazione privilegerà:
- correttezza;
- determinismo;
- confronto con una versione CPU di riferimento;
- verificabilità dei risultati.

37
modules/mutual-reachability/README.md Normal file
View file

@ -0,0 +1,37 @@
# Mutual Reachability
Modulo futuro per convertire il grafo kNN in un grafo pesato utilizzabile
da HDBSCAN.
## Core distance
Con `min_samples=32`, la core distance di un punto corrisponde alla
distanza dal suo trentaduesimo vicino.
## Mutual reachability distance
```text
mrd(i,j) =
max(
core_distance(i),
core_distance(j),
distance(i,j)
)
```
## Input previsto
```text
indices[N][32]
scores[N][32]
```
## Output previsto
Un insieme di archi candidati:
```text
source
destination
mutual_reachability_weight
```

855
src/face_engine.cc Normal file
View file

@ -0,0 +1,855 @@
#include <algorithm>
#include <cmath>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <iostream>
#include <string>
#include <vector>
#define STB_IMAGE_IMPLEMENTATION
#include "stb_image.h"
#include "rknn_api.h"
#include "face_engine.h"
static constexpr int SCRFD_SIZE = 640;
static constexpr int SFACE_SIZE = 112;
static constexpr int EMBED_DIM = 128;
static uint16_t float_to_fp16(float value)
{
uint32_t bits;
std::memcpy(&bits, &value, sizeof(bits));
uint32_t sign = (bits >> 31) & 1;
int exp = ((bits >> 23) & 0xff) - 127;
uint32_t mant = bits & 0x7fffff;
if (exp == 128) {
return (sign << 15) | 0x7c00;
}
if (exp > 15) {
return (sign << 15) | 0x7c00;
}
if (exp < -14) {
if (exp < -24)
return sign << 15;
mant |= 0x800000;
int shift = -exp - 14;
uint16_t m = mant >> (shift + 13);
return (sign << 15) | m;
}
uint16_t h_exp = (uint16_t)(exp + 15);
uint16_t h_mant = (uint16_t)(mant >> 13);
return (sign << 15) | (h_exp << 10) | h_mant;
}
static float fp16_to_float(uint16_t h)
{
uint32_t sign = (h >> 15) & 1;
uint32_t exp = (h >> 10) & 0x1f;
uint32_t mant = h & 0x3ff;
uint32_t bits;
if (exp == 0) {
if (mant == 0) {
bits = sign << 31;
} else {
float v = mant / 1024.0f;
v = std::ldexp(v, -14);
return sign ? -v : v;
}
} else if (exp == 31) {
bits = (sign << 31) | 0x7f800000 | (mant << 13);
} else {
uint32_t fexp = exp - 15 + 127;
bits = (sign << 31) | (fexp << 23) | (mant << 13);
}
float v;
std::memcpy(&v, &bits, sizeof(v));
return v;
}
static float iou(const Face& a, const Face& b)
{
float xx1 = std::max(a.x1, b.x1);
float yy1 = std::max(a.y1, b.y1);
float xx2 = std::min(a.x2, b.x2);
float yy2 = std::min(a.y2, b.y2);
float w = std::max(0.0f, xx2 - xx1);
float h = std::max(0.0f, yy2 - yy1);
float inter = w * h;
float area_a = std::max(0.0f, a.x2-a.x1) *
std::max(0.0f, a.y2-a.y1);
float area_b = std::max(0.0f, b.x2-b.x1) *
std::max(0.0f, b.y2-b.y1);
return inter / (area_a + area_b - inter + 1e-6f);
}
static void nms(std::vector<Face>& faces, float threshold)
{
std::sort(faces.begin(), faces.end(),
[](const Face& a, const Face& b) {
return a.score > b.score;
});
std::vector<Face> result;
for (const auto& f : faces) {
bool keep = true;
for (const auto& r : result) {
if (iou(f, r) > threshold) {
keep = false;
break;
}
}
if (keep)
result.push_back(f);
}
faces.swap(result);
}
struct Image {
int w = 0;
int h = 0;
std::vector<uint8_t> rgb;
};
static bool normalize_rotation(
int rotation_degrees,
int& normalized_rotation)
{
int value = rotation_degrees % 360;
if (value < 0)
value += 360;
if (
value != 0
&& value != 90
&& value != 180
&& value != 270
) {
return false;
}
normalized_rotation = value;
return true;
}
static Image rotate_image(
const Image& source,
int rotation_degrees)
{
if (rotation_degrees == 0)
return source;
Image destination;
if (
rotation_degrees == 90
|| rotation_degrees == 270
) {
destination.w = source.h;
destination.h = source.w;
} else {
destination.w = source.w;
destination.h = source.h;
}
destination.rgb.resize(
static_cast<std::size_t>(destination.w)
* static_cast<std::size_t>(destination.h)
* 3);
for (int source_y = 0; source_y < source.h; ++source_y) {
for (int source_x = 0; source_x < source.w; ++source_x) {
int destination_x = 0;
int destination_y = 0;
switch (rotation_degrees) {
case 90:
destination_x = source.h - 1 - source_y;
destination_y = source_x;
break;
case 180:
destination_x = source.w - 1 - source_x;
destination_y = source.h - 1 - source_y;
break;
case 270:
destination_x = source_y;
destination_y = source.w - 1 - source_x;
break;
default:
destination_x = source_x;
destination_y = source_y;
break;
}
const std::size_t source_offset =
(
static_cast<std::size_t>(source_y)
* static_cast<std::size_t>(source.w)
+ static_cast<std::size_t>(source_x)
) * 3;
const std::size_t destination_offset =
(
static_cast<std::size_t>(destination_y)
* static_cast<std::size_t>(destination.w)
+ static_cast<std::size_t>(destination_x)
) * 3;
destination.rgb[destination_offset + 0] =
source.rgb[source_offset + 0];
destination.rgb[destination_offset + 1] =
source.rgb[source_offset + 1];
destination.rgb[destination_offset + 2] =
source.rgb[source_offset + 2];
}
}
return destination;
}
static bool load_image(const std::string& path, Image& img)
{
int c = 0;
unsigned char* p =
stbi_load(path.c_str(), &img.w, &img.h, &c, 3);
if (!p) {
std::cerr << "Errore caricamento: " << path << "\n";
return false;
}
img.rgb.assign(p, p + img.w * img.h * 3);
stbi_image_free(p);
return true;
}
static void resize_bilinear_rgb(
const Image& src,
std::vector<uint8_t>& dst,
int dw,
int dh)
{
dst.resize(dw * dh * 3);
float sx = (float)src.w / dw;
float sy = (float)src.h / dh;
for (int y = 0; y < dh; ++y) {
float fy = (y + 0.5f) * sy - 0.5f;
int y0 = (int)std::floor(fy);
float wy = fy - y0;
if (y0 < 0) {
y0 = 0;
wy = 0;
}
int y1 = std::min(y0 + 1, src.h - 1);
for (int x = 0; x < dw; ++x) {
float fx = (x + 0.5f) * sx - 0.5f;
int x0 = (int)std::floor(fx);
float wx = fx - x0;
if (x0 < 0) {
x0 = 0;
wx = 0;
}
int x1 = std::min(x0 + 1, src.w - 1);
for (int c = 0; c < 3; ++c) {
float p00 = src.rgb[(y0*src.w+x0)*3+c];
float p01 = src.rgb[(y0*src.w+x1)*3+c];
float p10 = src.rgb[(y1*src.w+x0)*3+c];
float p11 = src.rgb[(y1*src.w+x1)*3+c];
float v =
p00 * (1-wx) * (1-wy) +
p01 * wx * (1-wy) +
p10 * (1-wx) * wy +
p11 * wx * wy;
dst[(y*dw+x)*3+c] =
(uint8_t)std::clamp((int)std::round(v), 0, 255);
}
}
}
}
class Scrfd {
public:
rknn_context ctx = 0;
bool init(const char* model)
{
FILE* fp = fopen(model, "rb");
if (!fp) {
perror(model);
return false;
}
fseek(fp, 0, SEEK_END);
size_t size = ftell(fp);
fseek(fp, 0, SEEK_SET);
std::vector<uint8_t> data(size);
fread(data.data(), 1, size, fp);
fclose(fp);
int ret = rknn_init(&ctx, data.data(), size, 0, nullptr);
if (ret != 0) {
std::cerr << "rknn_init SCRFD failed: " << ret << "\n";
return false;
}
return true;
}
std::vector<Face> detect(const Image& image)
{
float scale =
std::min(640.0f / image.w, 640.0f / image.h);
int rw = (int)std::round(image.w * scale);
int rh = (int)std::round(image.h * scale);
std::vector<uint8_t> resized;
resize_bilinear_rgb(image, resized, rw, rh);
std::vector<uint16_t> input(
SCRFD_SIZE * SCRFD_SIZE * 3);
for (int y = 0; y < rh; ++y) {
for (int x = 0; x < rw; ++x) {
int dst = (y * SCRFD_SIZE + x) * 3;
int src = (y * rw + x) * 3;
for (int c = 0; c < 3; ++c) {
float v =
((float)resized[src+c] - 127.5f) / 128.0f;
input[dst+c] = float_to_fp16(v);
}
}
}
rknn_input in{};
in.index = 0;
in.type = RKNN_TENSOR_FLOAT16;
in.fmt = RKNN_TENSOR_NHWC;
in.size = input.size() * sizeof(uint16_t);
in.buf = input.data();
in.pass_through = 1;
int ret = rknn_inputs_set(ctx, 1, &in);
if (ret != 0) {
std::cerr << "SCRFD inputs_set failed\n";
return {};
}
ret = rknn_run(ctx, nullptr);
if (ret != 0) {
std::cerr << "SCRFD run failed\n";
return {};
}
rknn_output outputs[9]{};
for (int i = 0; i < 9; ++i) {
outputs[i].want_float = 0;
}
ret = rknn_outputs_get(ctx, 9, outputs, nullptr);
if (ret != 0) {
std::cerr << "SCRFD outputs_get failed\n";
return {};
}
const int strides[3] = {8, 16, 32};
const int counts[3] = {12800, 3200, 800};
std::vector<Face> faces;
const float score_threshold = 0.50f;
for (int level = 0; level < 3; ++level) {
const int stride = strides[level];
const int count = counts[level];
const uint16_t* scores =
(const uint16_t*)outputs[level].buf;
const uint16_t* bbox =
(const uint16_t*)outputs[3 + level].buf;
const uint16_t* kps =
(const uint16_t*)outputs[6 + level].buf;
int feat_w = SCRFD_SIZE / stride;
for (int i = 0; i < count; ++i) {
float score = fp16_to_float(scores[i]);
if (score < score_threshold)
continue;
int anchor = i % 2;
int p = i / 2;
int gx = p % feat_w;
int gy = p / feat_w;
float cx = gx * stride;
float cy = gy * stride;
/*
* AnchorGenerator con due anchor:
* centers duplicati sullo stesso punto.
*/
(void)anchor;
float l = fp16_to_float(bbox[i*4+0]) * stride;
float t = fp16_to_float(bbox[i*4+1]) * stride;
float r = fp16_to_float(bbox[i*4+2]) * stride;
float b = fp16_to_float(bbox[i*4+3]) * stride;
Face f;
f.score = score;
f.x1 = (cx - l) / scale;
f.y1 = (cy - t) / scale;
f.x2 = (cx + r) / scale;
f.y2 = (cy + b) / scale;
for (int j = 0; j < 5; ++j) {
float x =
cx + fp16_to_float(kps[i*10+j*2])
* stride;
float y =
cy + fp16_to_float(kps[i*10+j*2+1])
* stride;
f.kps[j].x = x / scale;
f.kps[j].y = y / scale;
}
f.x1 = std::clamp(f.x1, 0.0f, (float)image.w);
f.y1 = std::clamp(f.y1, 0.0f, (float)image.h);
f.x2 = std::clamp(f.x2, 0.0f, (float)image.w);
f.y2 = std::clamp(f.y2, 0.0f, (float)image.h);
faces.push_back(f);
}
}
rknn_outputs_release(ctx, 9, outputs);
nms(faces, 0.45f);
return faces;
}
~Scrfd()
{
if (ctx)
rknn_destroy(ctx);
}
};
static const Point sface_template[5] = {
{38.2946f, 51.6963f},
{73.5318f, 51.5014f},
{56.0252f, 71.7366f},
{41.5493f, 92.3655f},
{70.7299f, 92.2041f}
};
/*
* Trova la trasformazione similarity:
*
* x' = a*x - b*y + tx
* y' = b*x + a*y + ty
*/
static void similarity_transform(
const Point src[5],
float& a,
float& b,
float& tx,
float& ty)
{
double sx = 0, sy = 0;
double dx = 0, dy = 0;
for (int i = 0; i < 5; ++i) {
sx += src[i].x;
sy += src[i].y;
dx += sface_template[i].x;
dy += sface_template[i].y;
}
sx /= 5;
sy /= 5;
dx /= 5;
dy /= 5;
double num_a = 0;
double num_b = 0;
double den = 0;
for (int i = 0; i < 5; ++i) {
double x = src[i].x - sx;
double y = src[i].y - sy;
double X = sface_template[i].x - dx;
double Y = sface_template[i].y - dy;
num_a += x*X + y*Y;
num_b += x*Y - y*X;
den += x*x + y*y;
}
a = (float)(num_a / den);
b = (float)(num_b / den);
tx = (float)(dx - a*sx + b*sy);
ty = (float)(dy - b*sx - a*sy);
}
static std::vector<uint16_t> align_face(
const Image& image,
const Face& face)
{
float a, b, tx, ty;
similarity_transform(face.kps, a, b, tx, ty);
/*
* Inversa della similarity transform.
*
* src = A^-1 * (dst - t)
*/
float denom = a*a + b*b;
std::vector<uint16_t> output(
SFACE_SIZE * SFACE_SIZE * 3);
for (int y = 0; y < SFACE_SIZE; ++y) {
for (int x = 0; x < SFACE_SIZE; ++x) {
float X = x - tx;
float Y = y - ty;
float sx =
(a*X + b*Y) / denom;
float sy =
(-b*X + a*Y) / denom;
int x0 = (int)std::floor(sx);
int y0 = (int)std::floor(sy);
float wx = sx - x0;
float wy = sy - y0;
uint8_t rgb[3] = {0,0,0};
if (x0 >= 0 &&
y0 >= 0 &&
x0 + 1 < image.w &&
y0 + 1 < image.h) {
int x1 = x0 + 1;
int y1 = y0 + 1;
for (int c = 0; c < 3; ++c) {
float p00 =
image.rgb[(y0*image.w+x0)*3+c];
float p01 =
image.rgb[(y0*image.w+x1)*3+c];
float p10 =
image.rgb[(y1*image.w+x0)*3+c];
float p11 =
image.rgb[(y1*image.w+x1)*3+c];
float v =
p00*(1-wx)*(1-wy) +
p01*wx*(1-wy) +
p10*(1-wx)*wy +
p11*wx*wy;
rgb[c] =
(uint8_t)std::clamp(
(int)std::round(v), 0, 255);
}
}
int idx = (y*SFACE_SIZE+x)*3;
/*
* SFace OpenCV:
* blobFromImage(..., swapRB=true, scalefactor=1)
*
* Noi partiamo già da RGB, quindi non facciamo
* nessun ulteriore swap e nessuna normalizzazione.
*/
output[idx+0] = float_to_fp16(rgb[0]);
output[idx+1] = float_to_fp16(rgb[1]);
output[idx+2] = float_to_fp16(rgb[2]);
}
}
return output;
}
class SFace {
public:
rknn_context ctx = 0;
bool init(const char* model)
{
FILE* fp = fopen(model, "rb");
if (!fp) {
perror(model);
return false;
}
fseek(fp, 0, SEEK_END);
size_t size = ftell(fp);
fseek(fp, 0, SEEK_SET);
std::vector<uint8_t> data(size);
fread(data.data(), 1, size, fp);
fclose(fp);
int ret = rknn_init(&ctx, data.data(), size, 0, nullptr);
if (ret != 0) {
std::cerr << "rknn_init SFace failed: "
<< ret << "\n";
return false;
}
return true;
}
std::vector<float> feature(
const std::vector<uint16_t>& input)
{
rknn_input in{};
in.index = 0;
in.type = RKNN_TENSOR_FLOAT16;
in.fmt = RKNN_TENSOR_NHWC;
in.size = input.size() * sizeof(uint16_t);
in.buf = (void*)input.data();
in.pass_through = 1;
int ret = rknn_inputs_set(ctx, 1, &in);
if (ret != 0) {
std::cerr << "SFace inputs_set failed\n";
return {};
}
ret = rknn_run(ctx, nullptr);
if (ret != 0) {
std::cerr << "SFace run failed\n";
return {};
}
rknn_output output{};
output.want_float = 0;
ret = rknn_outputs_get(
ctx, 1, &output, nullptr);
if (ret != 0) {
std::cerr << "SFace outputs_get failed\n";
return {};
}
const uint16_t* p =
(const uint16_t*)output.buf;
std::vector<float> emb(EMBED_DIM);
for (int i = 0; i < EMBED_DIM; ++i)
emb[i] = fp16_to_float(p[i]);
rknn_outputs_release(ctx, 1, &output);
/*
* L2 normalization.
*/
double norm = 0;
for (float v : emb)
norm += (double)v * v;
norm = std::sqrt(norm);
if (norm > 0) {
for (float& v : emb)
v /= (float)norm;
}
return emb;
}
~SFace()
{
if (ctx)
rknn_destroy(ctx);
}
};
static std::vector<Face> process_image(
const Image& image,
Scrfd& detector,
SFace& recognizer)
{
auto faces = detector.detect(image);
for (auto& face : faces) {
auto aligned = align_face(image, face);
face.embedding =
recognizer.feature(aligned);
}
return faces;
}
struct FaceEngine::Impl {
Scrfd detector;
SFace recognizer;
bool initialized = false;
};
FaceEngine::FaceEngine()
: impl_(new Impl())
{
}
FaceEngine::~FaceEngine()
{
delete impl_;
}
bool FaceEngine::init(
const std::string& scrfd_model_path,
const std::string& sface_model_path)
{
if (impl_->initialized)
return true;
if (!impl_->detector.init(scrfd_model_path.c_str()))
return false;
if (!impl_->recognizer.init(sface_model_path.c_str()))
return false;
impl_->initialized = true;
return true;
}
bool FaceEngine::process_image(
const std::string& image_path,
std::vector<Face>& faces,
int& image_width,
int& image_height,
int rotation_degrees)
{
faces.clear();
image_width = 0;
image_height = 0;
if (!impl_->initialized) {
std::cerr << "FaceEngine non inizializzato\n";
return false;
}
int normalized_rotation = 0;
if (!normalize_rotation(
rotation_degrees,
normalized_rotation)) {
std::cerr
<< "Rotazione non valida: "
<< rotation_degrees
<< ". Valori ammessi: 0, 90, 180, 270\n";
return false;
}
Image source_image;
if (!load_image(image_path, source_image))
return false;
Image oriented_image = rotate_image(
source_image,
normalized_rotation);
image_width = oriented_image.w;
image_height = oriented_image.h;
faces = ::process_image(
oriented_image,
impl_->detector,
impl_->recognizer);
for (const Face& face : faces) {
if (face.embedding.size() != EMBED_DIM) {
std::cerr
<< "Embedding non valido per "
<< image_path
<< ": attesi "
<< EMBED_DIM
<< " valori, trovati "
<< face.embedding.size()
<< "\n";
faces.clear();
return false;
}
}
return true;
}

781
src/face_knn.cc Normal file
View file

@ -0,0 +1,781 @@
#include "knn_opencl.h"
#include <cstddef>
#include <cstdint>
#include <cstdlib>
#include <fstream>
#include <iomanip>
#include <iostream>
#include <limits>
#include <stdexcept>
#include <string>
#include <vector>
#include <cstdio>
#include <unistd.h>
namespace {
class ScopedStdoutToStderr {
public:
ScopedStdoutToStderr()
{
std::cout.flush();
std::fflush(stdout);
saved_stdout_ =
::dup(STDOUT_FILENO);
if (saved_stdout_ < 0) {
throw std::runtime_error(
"Impossibile salvare stdout"
);
}
if (
::dup2(
STDERR_FILENO,
STDOUT_FILENO
) < 0
) {
::close(saved_stdout_);
saved_stdout_ = -1;
throw std::runtime_error(
"Impossibile deviare stdout"
);
}
}
~ScopedStdoutToStderr()
{
if (saved_stdout_ < 0) {
return;
}
std::cout.flush();
std::fflush(stdout);
::dup2(
saved_stdout_,
STDOUT_FILENO
);
::close(saved_stdout_);
}
ScopedStdoutToStderr(
const ScopedStdoutToStderr&
) = delete;
ScopedStdoutToStderr& operator=(
const ScopedStdoutToStderr&
) = delete;
private:
int saved_stdout_ = -1;
};
struct Arguments {
std::string queries_path;
std::string database_path;
std::string self_map_path;
std::string backend = "cpu";
std::size_t query_count = 0;
std::size_t database_count = 0;
std::uint32_t dimension = 128;
std::uint32_t top_k = 1;
float threshold = 0.50f;
};
void print_usage(
const char* program)
{
std::cerr
<< "Uso:\n"
<< " "
<< program
<< " --queries FILE"
<< " --database FILE"
<< " --query-count N"
<< " --database-count N"
<< " --dimension N"
<< " --top-k N"
<< " --backend cpu"
<< " [--threshold SCORE]"
<< " [--self-map FILE]"
<< "\n";
}
std::string require_value(
int argc,
char** argv,
int& index)
{
if (index + 1 >= argc) {
throw std::invalid_argument(
std::string(
"Valore mancante per "
)
+ argv[index]
);
}
index += 1;
return argv[index];
}
std::size_t parse_size(
const std::string& text,
const char* name)
{
std::size_t consumed = 0;
const auto value =
std::stoull(
text,
&consumed,
10
);
if (
consumed != text.size()
|| value == 0
) {
throw std::invalid_argument(
std::string(name)
+ " non valido: "
+ text
);
}
return static_cast<std::size_t>(
value
);
}
std::uint32_t parse_u32(
const std::string& text,
const char* name)
{
const std::size_t value =
parse_size(
text,
name
);
if (
value
> std::numeric_limits<
std::uint32_t
>::max()
) {
throw std::invalid_argument(
std::string(name)
+ " supera uint32"
);
}
return static_cast<std::uint32_t>(
value
);
}
float parse_float(
const std::string& text,
const char* name)
{
std::size_t consumed = 0;
const float value =
std::stof(
text,
&consumed
);
if (consumed != text.size()) {
throw std::invalid_argument(
std::string(name)
+ " non valido: "
+ text
);
}
return value;
}
Arguments parse_arguments(
int argc,
char** argv)
{
Arguments arguments;
for (
int index = 1;
index < argc;
++index
) {
const std::string option =
argv[index];
if (
option == "--help"
|| option == "-h"
) {
print_usage(argv[0]);
std::exit(0);
}
if (option == "--queries") {
arguments.queries_path =
require_value(
argc,
argv,
index
);
} else if (
option == "--database"
) {
arguments.database_path =
require_value(
argc,
argv,
index
);
} else if (
option == "--self-map"
) {
arguments.self_map_path =
require_value(
argc,
argv,
index
);
} else if (
option == "--backend"
) {
arguments.backend =
require_value(
argc,
argv,
index
);
} else if (
option == "--query-count"
) {
arguments.query_count =
parse_size(
require_value(
argc,
argv,
index
),
"query-count"
);
} else if (
option == "--database-count"
) {
arguments.database_count =
parse_size(
require_value(
argc,
argv,
index
),
"database-count"
);
} else if (
option == "--dimension"
) {
arguments.dimension =
parse_u32(
require_value(
argc,
argv,
index
),
"dimension"
);
} else if (
option == "--top-k"
) {
arguments.top_k =
parse_u32(
require_value(
argc,
argv,
index
),
"top-k"
);
} else if (
option == "--threshold"
) {
arguments.threshold =
parse_float(
require_value(
argc,
argv,
index
),
"threshold"
);
} else {
throw std::invalid_argument(
"Opzione sconosciuta: "
+ option
);
}
}
if (
arguments.queries_path.empty()
|| arguments.database_path.empty()
|| arguments.query_count == 0
|| arguments.database_count == 0
) {
throw std::invalid_argument(
"Argomenti obbligatori mancanti"
);
}
if (
arguments.backend != "cpu"
&& arguments.backend != "opencl"
) {
throw std::invalid_argument(
"Backend non supportato: "
+ arguments.backend
);
}
return arguments;
}
std::vector<float> read_float_file(
const std::string& path,
std::size_t expected_values)
{
std::ifstream input(
path,
std::ios::binary
);
if (!input) {
throw std::runtime_error(
"Impossibile aprire: "
+ path
);
}
input.seekg(
0,
std::ios::end
);
const std::streamoff byte_count =
input.tellg();
input.seekg(
0,
std::ios::beg
);
const auto expected_bytes =
static_cast<std::streamoff>(
expected_values
* sizeof(float)
);
if (byte_count != expected_bytes) {
throw std::runtime_error(
"Dimensione file incoerente: "
+ path
+ ", byte="
+ std::to_string(byte_count)
+ ", attesi="
+ std::to_string(
expected_bytes
)
);
}
std::vector<float> values(
expected_values
);
input.read(
reinterpret_cast<char*>(
values.data()
),
byte_count
);
if (!input) {
throw std::runtime_error(
"Lettura incompleta: "
+ path
);
}
return values;
}
std::vector<std::int64_t> read_self_map(
const std::string& path,
std::size_t query_count)
{
std::vector<std::int64_t> values(
query_count,
-1
);
if (path.empty()) {
return values;
}
std::ifstream input(path);
if (!input) {
throw std::runtime_error(
"Impossibile aprire self-map: "
+ path
);
}
std::size_t index = 0;
std::int64_t value = -1;
while (input >> value) {
if (index >= query_count) {
throw std::runtime_error(
"Self-map contiene "
"troppe righe"
);
}
values[index] = value;
index += 1;
}
if (index != query_count) {
throw std::runtime_error(
"Self-map contiene "
+ std::to_string(index)
+ " righe, attese "
+ std::to_string(
query_count
)
);
}
return values;
}
void write_json(
const face_knn::SearchOutput& output,
const std::vector<std::int64_t>& self_map,
float threshold)
{
std::size_t recognized_count = 0;
std::cout
<< std::setprecision(9)
<< "{\n"
<< " \"backend\": \""
<< output.backend
<< "\",\n"
<< " \"device_name\": \""
<< output.device_name
<< "\",\n"
<< " \"query_count\": "
<< output.query_count
<< ",\n"
<< " \"database_count\": "
<< output.database_count
<< ",\n"
<< " \"dimension\": "
<< output.dimension
<< ",\n"
<< " \"top_k\": "
<< output.top_k
<< ",\n"
<< " \"threshold\": "
<< threshold
<< ",\n"
<< " \"leave_one_out\": "
<< (
self_map.empty()
? "false"
: "true"
)
<< ",\n"
<< " \"elapsed_ms\": "
<< output.elapsed_ms
<< ",\n"
<< " \"results\": [\n";
for (
std::size_t query_index = 0;
query_index < output.query_count;
++query_index
) {
auto neighbors =
face_knn::query_neighbors(
output,
query_index
);
const std::int64_t excluded =
self_map[query_index];
if (excluded >= 0) {
neighbors =
face_knn::exclude_neighbor(
neighbors,
static_cast<
std::uint32_t
>(excluded)
);
}
if (neighbors.empty()) {
throw std::runtime_error(
"Nessun vicino disponibile "
"dopo leave-one-out"
);
}
const auto& best =
neighbors.front();
const bool recognized =
best.score >= threshold;
if (recognized) {
recognized_count += 1;
}
std::cout
<< " {\n"
<< " \"query_index\": "
<< query_index
<< ",\n"
<< " \"excluded_index\": ";
if (excluded < 0) {
std::cout << "null";
} else {
std::cout << excluded;
}
std::cout
<< ",\n"
<< " \"recognized\": "
<< (
recognized
? "true"
: "false"
)
<< ",\n"
<< " \"best_index\": "
<< best.index
<< ",\n"
<< " \"best_score\": "
<< best.score
<< ",\n"
<< " \"neighbors\": [\n";
for (
std::size_t position = 0;
position < neighbors.size();
++position
) {
const auto& neighbor =
neighbors[position];
std::cout
<< " {"
<< "\"index\": "
<< neighbor.index
<< ", \"score\": "
<< neighbor.score
<< "}";
if (
position + 1
< neighbors.size()
) {
std::cout << ",";
}
std::cout << "\n";
}
std::cout
<< " ]\n"
<< " }";
if (
query_index + 1
< output.query_count
) {
std::cout << ",";
}
std::cout << "\n";
}
const std::size_t below_count =
output.query_count
- recognized_count;
const double coverage =
output.query_count > 0
? (
static_cast<double>(
recognized_count
)
/ static_cast<double>(
output.query_count
)
* 100.0
)
: 0.0;
std::cout
<< " ],\n"
<< " \"recognized_count\": "
<< recognized_count
<< ",\n"
<< " \"below_threshold_count\": "
<< below_count
<< ",\n"
<< " \"coverage_percent\": "
<< coverage
<< "\n"
<< "}\n";
}
} // namespace
int main(
int argc,
char** argv)
{
try {
const Arguments arguments =
parse_arguments(
argc,
argv
);
const std::size_t query_values =
arguments.query_count
* arguments.dimension;
const std::size_t database_values =
arguments.database_count
* arguments.dimension;
const auto queries =
read_float_file(
arguments.queries_path,
query_values
);
const auto database =
read_float_file(
arguments.database_path,
database_values
);
const auto self_map =
read_self_map(
arguments.self_map_path,
arguments.query_count
);
face_knn::SearchOptions options;
options.dimension =
arguments.dimension;
options.top_k =
arguments.top_k;
options.kernel_path =
"kernels/knn_float4_topk.cl";
face_knn::SearchOutput output;
if (
arguments.backend == "opencl"
) {
/*
* Il runtime Mali scrive la propria versione
* direttamente su stdout. Durante OpenCL
* deviamo stdout su stderr, affinché l'output
* JSON resti puro.
*/
ScopedStdoutToStderr redirect;
output =
face_knn::search_opencl(
database,
queries,
arguments.database_count,
arguments.query_count,
options
);
} else {
output =
face_knn::search_cpu(
database,
queries,
arguments.database_count,
arguments.query_count,
options
);
}
write_json(
output,
self_map,
arguments.threshold
);
return 0;
} catch (const std::exception& error) {
std::cerr
<< "[face_knn] errore: "
<< error.what()
<< "\n";
print_usage(argv[0]);
return 1;
}
}

144
src/face_recognition.cc Normal file
View file

@ -0,0 +1,144 @@
#include <cstddef>
#include <iostream>
#include <string>
#include <vector>
#include "face_engine.h"
#include "face_config.h"
static constexpr float MATCH_THRESHOLD = 0.363f;
static float cosine(
const std::vector<float>& a,
const std::vector<float>& b)
{
if (a.size() != b.size())
return 0.0f;
float score = 0.0f;
for (std::size_t i = 0; i < a.size(); ++i)
score += a[i] * b[i];
return score;
}
static void print_faces(
const std::string& image_path,
const std::vector<Face>& faces)
{
std::cout << "\n" << image_path << "\n";
std::cout << "Faces: " << faces.size() << "\n";
for (std::size_t i = 0; i < faces.size(); ++i) {
const Face& face = faces[i];
std::cout
<< "\nFace " << i
<< " score=" << face.score
<< "\n bbox="
<< face.x1 << " "
<< face.y1 << " "
<< face.x2 << " "
<< face.y2
<< "\n";
std::cout << " landmarks:\n";
for (int j = 0; j < 5; ++j) {
std::cout
<< " "
<< j
<< ": "
<< face.kps[j].x
<< " "
<< face.kps[j].y
<< "\n";
}
}
}
int main(int argc, char** argv)
{
if (argc != 2 && argc != 3) {
std::cerr
<< "Uso:\n"
<< " " << argv[0]
<< " image.jpg\n"
<< " " << argv[0]
<< " reference.jpg query.jpg\n";
return 1;
}
const std::string root =
FACE_CLUSTERING_ROOT;
const std::string scrfd_model =
root + "/models/SCRFD_500M_KPS_640.rknn";
const std::string sface_model =
root + "/models/face_recognition_sface_2021dec.rknn";
FaceEngine engine;
if (!engine.init(scrfd_model, sface_model))
return 1;
std::vector<Face> faces1;
int image1_width = 0;
int image1_height = 0;
if (!engine.process_image(
argv[1],
faces1,
image1_width,
image1_height)) {
return 1;
}
print_faces(argv[1], faces1);
if (argc == 2)
return 0;
std::vector<Face> faces2;
int image2_width = 0;
int image2_height = 0;
if (!engine.process_image(
argv[2],
faces2,
image2_width,
image2_height)) {
return 1;
}
print_faces(argv[2], faces2);
std::cout
<< "\n========================================\n"
<< "COSINE SIMILARITY\n"
<< "========================================\n";
for (std::size_t i = 0; i < faces1.size(); ++i) {
for (std::size_t j = 0; j < faces2.size(); ++j) {
const float similarity = cosine(
faces1[i].embedding,
faces2[j].embedding);
std::cout
<< "ref[" << i
<< "] vs query[" << j
<< "] = "
<< similarity;
if (similarity >= MATCH_THRESHOLD)
std::cout << " MATCH";
std::cout << "\n";
}
}
return 0;
}

368
src/face_scan.cc Normal file
View file

@ -0,0 +1,368 @@
#include <cmath>
#include <cstddef>
#include <cstdlib>
#include <filesystem>
#include <iomanip>
#include <iostream>
#include <stdexcept>
#include <string>
#include <vector>
#include <unistd.h>
#include "face_engine.h"
static constexpr std::size_t EMBEDDING_SIZE = 128;
static std::filesystem::path executable_path()
{
std::vector<char> buffer(4096);
while (true) {
const ssize_t length = readlink(
"/proc/self/exe",
buffer.data(),
buffer.size() - 1);
if (length < 0) {
throw std::runtime_error(
"Impossibile leggere /proc/self/exe");
}
const std::size_t path_length =
static_cast<std::size_t>(length);
if (path_length < buffer.size() - 1) {
buffer[path_length] = '\0';
return std::filesystem::path(
buffer.data());
}
buffer.resize(buffer.size() * 2);
}
}
static std::filesystem::path application_root()
{
/*
* Struttura prevista:
*
* root/
* bin/face_scan
* lib/librknnrt.so
* models/<model>.rknn
*
* executable_path():
* root/bin/face_scan
*
* parent_path():
* root/bin
*
* parent_path().parent_path():
* root
*/
return executable_path()
.parent_path()
.parent_path();
}
static void write_json_string(const std::string& value)
{
std::cout << '"';
for (unsigned char ch : value) {
switch (ch) {
case '"':
std::cout << "\\\"";
break;
case '\\':
std::cout << "\\\\";
break;
case '\b':
std::cout << "\\b";
break;
case '\f':
std::cout << "\\f";
break;
case '\n':
std::cout << "\\n";
break;
case '\r':
std::cout << "\\r";
break;
case '\t':
std::cout << "\\t";
break;
default:
if (ch < 0x20) {
const char* hex = "0123456789abcdef";
std::cout
<< "\\u00"
<< hex[(ch >> 4) & 0x0f]
<< hex[ch & 0x0f];
} else {
std::cout << static_cast<char>(ch);
}
}
}
std::cout << '"';
}
static bool finite_face(const Face& face)
{
if (!std::isfinite(face.score) ||
!std::isfinite(face.x1) ||
!std::isfinite(face.y1) ||
!std::isfinite(face.x2) ||
!std::isfinite(face.y2)) {
return false;
}
for (const Point& point : face.kps) {
if (!std::isfinite(point.x) ||
!std::isfinite(point.y)) {
return false;
}
}
if (face.embedding.size() != EMBEDDING_SIZE)
return false;
for (float value : face.embedding) {
if (!std::isfinite(value))
return false;
}
return true;
}
static void write_face_json(
const Face& face,
std::size_t face_index)
{
const float width = face.x2 - face.x1;
const float height = face.y2 - face.y1;
std::cout << "{";
std::cout
<< "\"face_index\":"
<< face_index
<< ",";
std::cout
<< "\"score\":"
<< face.score
<< ",";
std::cout << "\"bbox\":{";
std::cout
<< "\"x\":"
<< face.x1
<< ",";
std::cout
<< "\"y\":"
<< face.y1
<< ",";
std::cout
<< "\"width\":"
<< width
<< ",";
std::cout
<< "\"height\":"
<< height;
std::cout << "},";
std::cout << "\"landmarks\":[";
for (int i = 0; i < 5; ++i) {
if (i != 0)
std::cout << ",";
std::cout
<< "["
<< face.kps[i].x
<< ","
<< face.kps[i].y
<< "]";
}
std::cout << "],";
std::cout << "\"embedding\":[";
for (std::size_t i = 0;
i < face.embedding.size();
++i) {
if (i != 0)
std::cout << ",";
std::cout << face.embedding[i];
}
std::cout << "]";
std::cout << "}";
}
int main(int argc, char** argv)
{
int rotation_degrees = 0;
const char* image_path = nullptr;
if (argc == 2) {
image_path = argv[1];
} else if (
argc == 4
&& std::string(argv[1]) == "--rotation"
) {
char* parse_end = nullptr;
const long parsed_rotation = std::strtol(
argv[2],
&parse_end,
10);
if (
parse_end == argv[2]
|| *parse_end != '\0'
|| parsed_rotation < -360000
|| parsed_rotation > 360000
) {
std::cerr
<< "Rotazione non valida: "
<< argv[2]
<< "\n";
return 2;
}
rotation_degrees =
static_cast<int>(parsed_rotation);
image_path = argv[3];
} else {
std::cerr
<< "Uso:\n"
<< " " << argv[0]
<< " image.jpg\n"
<< " " << argv[0]
<< " --rotation 90 image.jpg\n";
return 2;
}
std::filesystem::path root;
try {
root = application_root();
} catch (const std::exception& error) {
std::cerr
<< "Impossibile determinare la root applicativa: "
<< error.what()
<< "\n";
return 3;
}
const std::string scrfd_model =
(
root
/ "models"
/ "SCRFD_500M_KPS_640.rknn"
).string();
const std::string sface_model =
(
root
/ "models"
/ "face_recognition_sface_2021dec.rknn"
).string();
FaceEngine engine;
if (!engine.init(scrfd_model, sface_model)) {
std::cerr
<< "Impossibile inizializzare FaceEngine\n";
return 3;
}
std::vector<Face> faces;
int image_width = 0;
int image_height = 0;
if (!engine.process_image(
image_path,
faces,
image_width,
image_height,
rotation_degrees)) {
std::cerr
<< "Scansione fallita: "
<< image_path
<< "\n";
return 4;
}
for (std::size_t i = 0; i < faces.size(); ++i) {
if (!finite_face(faces[i])) {
std::cerr
<< "Risultato non valido per face_index="
<< i
<< "\n";
return 5;
}
}
std::cout << std::setprecision(9);
std::cout << "{";
std::cout << "\"image\":{";
std::cout << "\"path\":";
write_json_string(image_path);
std::cout
<< ",\"width\":"
<< image_width
<< ",\"height\":"
<< image_height
<< ",\"rotation\":"
<< rotation_degrees;
std::cout << "},";
std::cout << "\"faces\":[";
for (std::size_t i = 0; i < faces.size(); ++i) {
if (i != 0)
std::cout << ",";
write_face_json(faces[i], i);
}
std::cout << "]";
std::cout << "}\n";
return 0;
}

484
src/knn_opencl.cc Normal file
View file

@ -0,0 +1,484 @@
#include "knn_opencl.h"
#include <algorithm>
#include <chrono>
#include <cmath>
#include <limits>
#include <stdexcept>
#include <string>
#include <vector>
namespace face_knn {
SearchOutput search_opencl_impl(
const std::vector<float>& database,
const std::vector<float>& queries,
std::size_t database_count,
std::size_t query_count,
const SearchOptions& options);
namespace {
bool better_neighbor(
const Neighbor& left,
const Neighbor& right)
{
if (left.score != right.score) {
return left.score > right.score;
}
return left.index < right.index;
}
void validate_finite_values(
const std::vector<float>& values,
const char* name)
{
for (
std::size_t index = 0;
index < values.size();
++index
) {
if (!std::isfinite(values[index])) {
throw std::invalid_argument(
std::string(name)
+ " contiene un valore non finito"
);
}
}
}
std::uint32_t effective_top_k(
std::size_t database_count,
std::uint32_t requested_top_k)
{
return std::min(
requested_top_k,
static_cast<std::uint32_t>(
database_count
)
);
}
} // namespace
void validate_input(
const std::vector<float>& database,
const std::vector<float>& queries,
std::size_t database_count,
std::size_t query_count,
const SearchOptions& options)
{
if (options.dimension == 0) {
throw std::invalid_argument(
"dimension deve essere maggiore di zero"
);
}
if (options.top_k == 0) {
throw std::invalid_argument(
"top_k deve essere maggiore di zero"
);
}
if (options.lanes_per_query == 0) {
throw std::invalid_argument(
"lanes_per_query deve essere maggiore di zero"
);
}
if (options.chunk_size == 0) {
throw std::invalid_argument(
"chunk_size deve essere maggiore di zero"
);
}
if (database_count == 0) {
throw std::invalid_argument(
"database_count deve essere maggiore di zero"
);
}
if (query_count == 0) {
throw std::invalid_argument(
"query_count deve essere maggiore di zero"
);
}
if (
database_count
> std::numeric_limits<
std::uint32_t
>::max()
) {
throw std::invalid_argument(
"database_count supera uint32"
);
}
const std::size_t expected_database =
database_count
* static_cast<std::size_t>(
options.dimension
);
const std::size_t expected_queries =
query_count
* static_cast<std::size_t>(
options.dimension
);
if (database.size() != expected_database) {
throw std::invalid_argument(
"dimensione database incoerente"
);
}
if (queries.size() != expected_queries) {
throw std::invalid_argument(
"dimensione query incoerente"
);
}
validate_finite_values(
database,
"database"
);
validate_finite_values(
queries,
"queries"
);
}
SearchOutput search_cpu(
const std::vector<float>& database,
const std::vector<float>& queries,
std::size_t database_count,
std::size_t query_count,
const SearchOptions& options)
{
validate_input(
database,
queries,
database_count,
query_count,
options
);
const std::uint32_t result_top_k =
effective_top_k(
database_count,
options.top_k
);
SearchOutput output;
output.query_count = query_count;
output.database_count = database_count;
output.dimension = options.dimension;
output.top_k = result_top_k;
output.backend = "cpu";
output.device_name = "CPU";
output.neighbors.resize(
query_count
* static_cast<std::size_t>(
result_top_k
)
);
const auto started_at =
std::chrono::steady_clock::now();
for (
std::size_t query_index = 0;
query_index < query_count;
++query_index
) {
const float* query =
&queries[
query_index
* options.dimension
];
std::vector<Neighbor> candidates;
candidates.reserve(database_count);
for (
std::size_t database_index = 0;
database_index < database_count;
++database_index
) {
const float* database_vector =
&database[
database_index
* options.dimension
];
float score = 0.0f;
for (
std::uint32_t component = 0;
component < options.dimension;
++component
) {
score +=
database_vector[component]
* query[component];
}
candidates.push_back({
score,
static_cast<std::uint32_t>(
database_index
),
});
}
if (
result_top_k
< candidates.size()
) {
std::nth_element(
candidates.begin(),
candidates.begin()
+ result_top_k,
candidates.end(),
better_neighbor
);
}
std::sort(
candidates.begin(),
candidates.begin()
+ result_top_k,
better_neighbor
);
const std::size_t output_offset =
query_index
* result_top_k;
for (
std::uint32_t position = 0;
position < result_top_k;
++position
) {
output.neighbors[
output_offset + position
] = candidates[position];
}
}
const auto completed_at =
std::chrono::steady_clock::now();
output.elapsed_ms =
std::chrono::duration<
double,
std::milli
>(
completed_at - started_at
).count();
return output;
}
SearchOutput search_opencl(
const std::vector<float>& database,
const std::vector<float>& queries,
std::size_t database_count,
std::size_t query_count,
const SearchOptions& options)
{
return search_opencl_impl(
database,
queries,
database_count,
query_count,
options
);
}
std::vector<Neighbor> query_neighbors(
const SearchOutput& output,
std::size_t query_index)
{
if (query_index >= output.query_count) {
throw std::out_of_range(
"query_index fuori intervallo"
);
}
const std::size_t begin =
query_index
* output.top_k;
const std::size_t end =
begin
+ output.top_k;
if (end > output.neighbors.size()) {
throw std::runtime_error(
"layout SearchOutput incoerente"
);
}
return {
output.neighbors.begin() + begin,
output.neighbors.begin() + end,
};
}
std::vector<Neighbor> exclude_neighbor(
const std::vector<Neighbor>& neighbors,
std::uint32_t excluded_index)
{
std::vector<Neighbor> filtered;
filtered.reserve(neighbors.size());
for (const Neighbor& neighbor : neighbors) {
if (neighbor.index == invalid_index) {
continue;
}
if (neighbor.index == excluded_index) {
continue;
}
filtered.push_back(neighbor);
}
return filtered;
}
ValidationResult compare_outputs(
const SearchOutput& expected,
const SearchOutput& actual,
float score_tolerance,
float recognition_threshold)
{
if (score_tolerance < 0.0f) {
throw std::invalid_argument(
"score_tolerance non valida"
);
}
if (
expected.query_count
!= actual.query_count
|| expected.database_count
!= actual.database_count
|| expected.dimension
!= actual.dimension
|| expected.top_k
!= actual.top_k
|| expected.neighbors.size()
!= actual.neighbors.size()
) {
throw std::invalid_argument(
"Output kNN non confrontabili"
);
}
ValidationResult result;
result.positions =
expected.neighbors.size();
for (
std::size_t index = 0;
index < result.positions;
++index
) {
const Neighbor& expected_neighbor =
expected.neighbors[index];
const Neighbor& actual_neighbor =
actual.neighbors[index];
if (
expected_neighbor.index
== actual_neighbor.index
) {
result.exact_index_matches += 1;
}
const float error = std::fabs(
expected_neighbor.score
- actual_neighbor.score
);
result.maximum_score_error =
std::max(
result.maximum_score_error,
error
);
}
for (
std::size_t query_index = 0;
query_index < expected.query_count;
++query_index
) {
const auto expected_neighbors =
query_neighbors(
expected,
query_index
);
const auto actual_neighbors =
query_neighbors(
actual,
query_index
);
const bool expected_recognized =
!expected_neighbors.empty()
&& expected_neighbors.front().score
>= recognition_threshold;
const bool actual_recognized =
!actual_neighbors.empty()
&& actual_neighbors.front().score
>= recognition_threshold;
if (
expected_recognized
== actual_recognized
) {
result.decision_matches += 1;
}
}
result.exact_indices =
result.exact_index_matches
== result.positions;
result.decisions_equal =
result.decision_matches
== expected.query_count;
if (
result.maximum_score_error
> score_tolerance
) {
result.decisions_equal = false;
}
return result;
}
} // namespace face_knn

944
src/knn_opencl_backend.cc Normal file
View file

@ -0,0 +1,944 @@
#define CL_TARGET_OPENCL_VERSION 120
#include <CL/cl.h>
#include "knn_opencl.h"
#include <algorithm>
#include <chrono>
#include <cstdint>
#include <fstream>
#include <iostream>
#include <iterator>
#include <limits>
#include <stdexcept>
#include <string>
#include <vector>
namespace face_knn {
namespace {
void check_cl(
cl_int status,
const char* operation)
{
if (status != CL_SUCCESS) {
throw std::runtime_error(
std::string(operation)
+ ": OpenCL status "
+ std::to_string(status)
);
}
}
std::string device_string(
cl_device_id device,
cl_device_info property)
{
size_t size = 0;
check_cl(
clGetDeviceInfo(
device,
property,
0,
nullptr,
&size
),
"clGetDeviceInfo size"
);
std::string value(
size,
'\0'
);
check_cl(
clGetDeviceInfo(
device,
property,
size,
value.data(),
nullptr
),
"clGetDeviceInfo value"
);
while (
!value.empty()
&& value.back() == '\0'
) {
value.pop_back();
}
return value;
}
cl_device_id find_mali_device(
std::string& device_name)
{
cl_uint platform_count = 0;
check_cl(
clGetPlatformIDs(
0,
nullptr,
&platform_count
),
"clGetPlatformIDs count"
);
if (platform_count == 0) {
throw std::runtime_error(
"Nessuna piattaforma OpenCL"
);
}
std::vector<cl_platform_id> platforms(
platform_count
);
check_cl(
clGetPlatformIDs(
platform_count,
platforms.data(),
nullptr
),
"clGetPlatformIDs list"
);
cl_device_id fallback = nullptr;
std::string fallback_name;
for (
cl_platform_id platform
: platforms
) {
cl_uint device_count = 0;
const cl_int status =
clGetDeviceIDs(
platform,
CL_DEVICE_TYPE_GPU,
0,
nullptr,
&device_count
);
if (
status != CL_SUCCESS
|| device_count == 0
) {
continue;
}
std::vector<cl_device_id> devices(
device_count
);
check_cl(
clGetDeviceIDs(
platform,
CL_DEVICE_TYPE_GPU,
device_count,
devices.data(),
nullptr
),
"clGetDeviceIDs list"
);
for (
cl_device_id device
: devices
) {
const std::string name =
device_string(
device,
CL_DEVICE_NAME
);
const std::string vendor =
device_string(
device,
CL_DEVICE_VENDOR
);
const std::string full_name =
vendor + " / " + name;
if (fallback == nullptr) {
fallback = device;
fallback_name = full_name;
}
if (
name.find("Mali-G610")
!= std::string::npos
) {
device_name = full_name;
return device;
}
}
}
if (fallback != nullptr) {
device_name = fallback_name;
std::cerr
<< "[face_knn] Mali-G610 non trovata; "
<< "uso GPU OpenCL: "
<< fallback_name
<< "\n";
return fallback;
}
throw std::runtime_error(
"Nessuna GPU OpenCL disponibile"
);
}
std::string read_text_file(
const std::string& path)
{
std::ifstream input(path);
if (!input) {
throw std::runtime_error(
"Kernel OpenCL non leggibile: "
+ path
);
}
const std::string content{
std::istreambuf_iterator<char>(
input
),
std::istreambuf_iterator<char>()
};
if (content.empty()) {
throw std::runtime_error(
"Kernel OpenCL vuoto: "
+ path
);
}
return content;
}
bool better_neighbor(
const Neighbor& left,
const Neighbor& right)
{
if (left.score != right.score) {
return left.score > right.score;
}
return left.index < right.index;
}
Neighbor invalid_neighbor()
{
return {
-std::numeric_limits<float>::infinity(),
invalid_index,
};
}
void merge_top_k(
const std::vector<std::uint32_t>& chunk_indices,
const std::vector<float>& chunk_scores,
std::size_t chunk_offset,
std::size_t query_count,
std::uint32_t top_k,
std::vector<Neighbor>& global_result)
{
for (
std::size_t query_index = 0;
query_index < query_count;
++query_index
) {
const std::size_t result_offset =
query_index
* top_k;
std::vector<Neighbor> candidates;
candidates.reserve(
static_cast<std::size_t>(
top_k
) * 2
);
for (
std::uint32_t position = 0;
position < top_k;
++position
) {
const Neighbor& current =
global_result[
result_offset
+ position
];
if (
current.index
!= invalid_index
) {
candidates.push_back(
current
);
}
}
for (
std::uint32_t position = 0;
position < top_k;
++position
) {
const std::size_t chunk_position =
result_offset
+ position;
const std::uint32_t local_index =
chunk_indices[
chunk_position
];
if (
local_index
== invalid_index
) {
continue;
}
candidates.push_back({
chunk_scores[
chunk_position
],
static_cast<std::uint32_t>(
chunk_offset
+ local_index
),
});
}
std::sort(
candidates.begin(),
candidates.end(),
better_neighbor
);
for (
std::uint32_t position = 0;
position < top_k;
++position
) {
Neighbor value =
invalid_neighbor();
if (
position
< candidates.size()
) {
value =
candidates[position];
}
global_result[
result_offset
+ position
] = value;
}
}
}
struct OpenCLResources {
cl_context context = nullptr;
cl_command_queue queue = nullptr;
cl_program program = nullptr;
cl_kernel kernel = nullptr;
cl_mem database_buffer = nullptr;
cl_mem query_buffer = nullptr;
cl_mem index_buffer = nullptr;
cl_mem score_buffer = nullptr;
~OpenCLResources()
{
if (score_buffer != nullptr) {
clReleaseMemObject(
score_buffer
);
}
if (index_buffer != nullptr) {
clReleaseMemObject(
index_buffer
);
}
if (query_buffer != nullptr) {
clReleaseMemObject(
query_buffer
);
}
if (database_buffer != nullptr) {
clReleaseMemObject(
database_buffer
);
}
if (kernel != nullptr) {
clReleaseKernel(kernel);
}
if (program != nullptr) {
clReleaseProgram(program);
}
if (queue != nullptr) {
clReleaseCommandQueue(queue);
}
if (context != nullptr) {
clReleaseContext(context);
}
}
};
std::string build_log(
cl_program program,
cl_device_id device)
{
size_t size = 0;
clGetProgramBuildInfo(
program,
device,
CL_PROGRAM_BUILD_LOG,
0,
nullptr,
&size
);
std::string value(
size,
'\0'
);
if (size > 0) {
clGetProgramBuildInfo(
program,
device,
CL_PROGRAM_BUILD_LOG,
size,
value.data(),
nullptr
);
}
while (
!value.empty()
&& value.back() == '\0'
) {
value.pop_back();
}
return value;
}
} // namespace
SearchOutput search_opencl_impl(
const std::vector<float>& database,
const std::vector<float>& queries,
std::size_t database_count,
std::size_t query_count,
const SearchOptions& options)
{
validate_input(
database,
queries,
database_count,
query_count,
options
);
if (
query_count
> std::numeric_limits<
std::uint32_t
>::max()
) {
throw std::invalid_argument(
"query_count supera uint32"
);
}
const std::uint32_t top_k =
std::min(
options.top_k,
static_cast<std::uint32_t>(
database_count
)
);
const std::string kernel_path =
options.kernel_path.empty()
? "kernels/knn_float4_topk.cl"
: options.kernel_path;
const std::string kernel_source =
read_text_file(
kernel_path
);
std::string device_name;
const cl_device_id device =
find_mali_device(
device_name
);
std::cerr
<< "[face_knn] OpenCL GPU: "
<< device_name
<< "\n";
OpenCLResources resources;
cl_int status = CL_SUCCESS;
resources.context =
clCreateContext(
nullptr,
1,
&device,
nullptr,
nullptr,
&status
);
check_cl(
status,
"clCreateContext"
);
resources.queue =
clCreateCommandQueue(
resources.context,
device,
0,
&status
);
check_cl(
status,
"clCreateCommandQueue"
);
const char* source_pointer =
kernel_source.c_str();
const size_t source_size =
kernel_source.size();
resources.program =
clCreateProgramWithSource(
resources.context,
1,
&source_pointer,
&source_size,
&status
);
check_cl(
status,
"clCreateProgramWithSource"
);
const std::string build_options =
"-cl-fast-relaxed-math"
" -D DIM="
+ std::to_string(
options.dimension
)
+ " -D TOPK="
+ std::to_string(top_k)
+ " -D QUERY_COUNT="
+ std::to_string(query_count)
+ " -D LANES_PER_QUERY="
+ std::to_string(
options.lanes_per_query
);
status = clBuildProgram(
resources.program,
1,
&device,
build_options.c_str(),
nullptr,
nullptr
);
if (status != CL_SUCCESS) {
throw std::runtime_error(
"clBuildProgram fallito: "
+ build_log(
resources.program,
device
)
);
}
resources.kernel =
clCreateKernel(
resources.program,
"knn",
&status
);
check_cl(
status,
"clCreateKernel"
);
const std::size_t chunk_capacity =
std::min(
options.chunk_size,
database_count
);
resources.database_buffer =
clCreateBuffer(
resources.context,
CL_MEM_READ_ONLY,
chunk_capacity
* options.dimension
* sizeof(float),
nullptr,
&status
);
check_cl(
status,
"clCreateBuffer database"
);
resources.query_buffer =
clCreateBuffer(
resources.context,
CL_MEM_READ_ONLY
| CL_MEM_COPY_HOST_PTR,
queries.size()
* sizeof(float),
const_cast<float*>(
queries.data()
),
&status
);
check_cl(
status,
"clCreateBuffer queries"
);
const std::size_t result_count =
query_count
* static_cast<std::size_t>(
top_k
);
std::vector<std::uint32_t> chunk_indices(
result_count,
invalid_index
);
std::vector<float> chunk_scores(
result_count,
-std::numeric_limits<float>::infinity()
);
resources.index_buffer =
clCreateBuffer(
resources.context,
CL_MEM_WRITE_ONLY,
result_count
* sizeof(std::uint32_t),
nullptr,
&status
);
check_cl(
status,
"clCreateBuffer indices"
);
resources.score_buffer =
clCreateBuffer(
resources.context,
CL_MEM_WRITE_ONLY,
result_count
* sizeof(float),
nullptr,
&status
);
check_cl(
status,
"clCreateBuffer scores"
);
check_cl(
clSetKernelArg(
resources.kernel,
0,
sizeof(
resources.database_buffer
),
&resources.database_buffer
),
"clSetKernelArg database"
);
check_cl(
clSetKernelArg(
resources.kernel,
1,
sizeof(
resources.query_buffer
),
&resources.query_buffer
),
"clSetKernelArg queries"
);
check_cl(
clSetKernelArg(
resources.kernel,
2,
sizeof(
resources.index_buffer
),
&resources.index_buffer
),
"clSetKernelArg indices"
);
check_cl(
clSetKernelArg(
resources.kernel,
3,
sizeof(
resources.score_buffer
),
&resources.score_buffer
),
"clSetKernelArg scores"
);
SearchOutput output;
output.query_count =
query_count;
output.database_count =
database_count;
output.dimension =
options.dimension;
output.top_k =
top_k;
output.backend =
"opencl";
output.device_name =
device_name;
output.neighbors.assign(
result_count,
invalid_neighbor()
);
const std::size_t global_size =
query_count
* options.lanes_per_query;
const std::size_t local_size =
options.lanes_per_query;
const std::size_t chunk_count =
(
database_count
+ options.chunk_size
- 1
)
/ options.chunk_size;
std::cerr
<< "[face_knn] kernel_path="
<< kernel_path
<< "\n"
<< "[face_knn] build_options="
<< build_options
<< "\n"
<< "[face_knn] chunk_size="
<< chunk_capacity
<< "\n"
<< "[face_knn] chunks="
<< chunk_count
<< "\n";
const auto started_at =
std::chrono::steady_clock::now();
for (
std::size_t chunk_offset = 0;
chunk_offset < database_count;
chunk_offset += options.chunk_size
) {
const std::size_t chunk_size =
std::min(
options.chunk_size,
database_count
- chunk_offset
);
const float* chunk_pointer =
&database[
chunk_offset
* options.dimension
];
check_cl(
clEnqueueWriteBuffer(
resources.queue,
resources.database_buffer,
CL_TRUE,
0,
chunk_size
* options.dimension
* sizeof(float),
chunk_pointer,
0,
nullptr,
nullptr
),
"clEnqueueWriteBuffer database"
);
const std::uint32_t kernel_count =
static_cast<std::uint32_t>(
chunk_size
);
check_cl(
clSetKernelArg(
resources.kernel,
4,
sizeof(kernel_count),
&kernel_count
),
"clSetKernelArg database count"
);
check_cl(
clEnqueueNDRangeKernel(
resources.queue,
resources.kernel,
1,
nullptr,
&global_size,
&local_size,
0,
nullptr,
nullptr
),
"clEnqueueNDRangeKernel"
);
check_cl(
clFinish(
resources.queue
),
"clFinish"
);
check_cl(
clEnqueueReadBuffer(
resources.queue,
resources.index_buffer,
CL_TRUE,
0,
chunk_indices.size()
* sizeof(std::uint32_t),
chunk_indices.data(),
0,
nullptr,
nullptr
),
"clEnqueueReadBuffer indices"
);
check_cl(
clEnqueueReadBuffer(
resources.queue,
resources.score_buffer,
CL_TRUE,
0,
chunk_scores.size()
* sizeof(float),
chunk_scores.data(),
0,
nullptr,
nullptr
),
"clEnqueueReadBuffer scores"
);
merge_top_k(
chunk_indices,
chunk_scores,
chunk_offset,
query_count,
top_k,
output.neighbors
);
}
const auto completed_at =
std::chrono::steady_clock::now();
output.elapsed_ms =
std::chrono::duration<
double,
std::milli
>(
completed_at - started_at
).count();
return output;
}
} // namespace face_knn

24
test/README.md Normal file
View file

@ -0,0 +1,24 @@
# Immagini di test
Inserire in questa directory immagini locali per verificare:
- rilevamento dei volti;
- bounding box;
- landmark;
- generazione degli embedding;
- cosine similarity;
- output JSON.
Esempio:
```bash
./bin/face_recognition test/reference.jpg test/query.jpg
```
Oppure:
```bash
./bin/face_scan test/image.jpg
```
Le immagini sono escluse dal repository tramite `.gitignore`.

107
test/knn/test_knn_cpu.cc Normal file
View file

@ -0,0 +1,107 @@
#include "knn_opencl.h"
#include <cmath>
#include <iostream>
#include <vector>
int main()
{
const std::vector<float> database = {
1.0f, 0.0f,
0.0f, 1.0f,
0.70710678f, 0.70710678f,
};
const std::vector<float> queries = {
1.0f, 0.0f,
0.0f, 1.0f,
};
face_knn::SearchOptions options;
options.dimension = 2;
options.top_k = 2;
const auto output =
face_knn::search_cpu(
database,
queries,
3,
2,
options
);
const auto first =
face_knn::query_neighbors(
output,
0
);
const auto second =
face_knn::query_neighbors(
output,
1
);
if (
first.size() != 2
|| second.size() != 2
) {
std::cerr
<< "Numero vicini errato\n";
return 1;
}
if (
first[0].index != 0
|| second[0].index != 1
) {
std::cerr
<< "Top-1 errato\n";
return 1;
}
if (
std::fabs(first[0].score - 1.0f)
> 0.000001f
|| std::fabs(second[0].score - 1.0f)
> 0.000001f
) {
std::cerr
<< "Score top-1 errato\n";
return 1;
}
const auto filtered =
face_knn::exclude_neighbor(
first,
0
);
if (
filtered.empty()
|| filtered[0].index != 2
) {
std::cerr
<< "Leave-one-out errato\n";
return 1;
}
std::cout
<< "OK: backend CPU dinamico\n"
<< "query_count="
<< output.query_count
<< "\n"
<< "database_count="
<< output.database_count
<< "\n"
<< "top_k="
<< output.top_k
<< "\n";
return 0;
}

BIN
test/test.jpg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 2 MiB

BIN
test/test2.jpg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.8 MiB

BIN
test/test3.jpg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 2.5 MiB

BIN
test/test3f.jpg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 401 KiB

BIN
test/test4.jpg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.7 MiB

BIN
test/test5.jpg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 230 KiB

BIN
test/test6.jpg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 3 MiB