AbstractPhil commited on
Commit
da18c24
Β·
verified Β·
1 Parent(s): 27dcd53

Update main README

Browse files
Files changed (1) hide show
  1. README.md +57 -118
README.md CHANGED
@@ -9,132 +9,50 @@ tags:
9
  - cantor-routing
10
  - pentachoron
11
  - multi-scale
12
- datasets:
13
- - cifar100
14
- metrics:
15
- - accuracy
16
- model-index:
17
- - name: DavidBeans
18
- results:
19
- - task:
20
- type: image-classification
21
- name: Image Classification
22
- dataset:
23
- name: CIFAR-100
24
- type: cifar100
25
- metrics:
26
- - type: accuracy
27
- value: 70.05
28
- name: Top-1 Accuracy
29
  ---
30
 
31
- # At long last my goal of 70% accuracy cifar100 on a geometric vit barrier breached
32
 
33
- David's primary metrics were based on clip-vit (the original gated-david repo), and now geovit-david-beans can exist as a stand-in for a vit!
34
 
35
- This is my first legitimate example of a geovit that I can attest is nearly to the expectations of geometric encoding in a useful fashion.
 
36
 
37
- ITS NOT THERE YET, but it's a definite benchmark I've been trying to achieve for quite some time, so I'm glad this one finally worked.
38
-
39
- # πŸ’Ž DavidBeans: Unified Vision-to-Crystal Architecture
40
-
41
- DavidBeans combines **ViT-Beans** (Cantor-routed sparse attention) with **David** (multi-scale crystal classification) into a unified geometric deep learning architecture.
42
-
43
- ## Model Description
44
-
45
- This model implements several novel techniques:
46
-
47
- - **Hybrid Cantor Routing**: Combines fractal Cantor set distances with positional proximity for sparse attention patterns
48
- - **Pentachoron Experts**: 5-vertex simplex structure with Cayley-Menger geometric regularization
49
- - **Multi-Scale Crystal Projection**: Projects features to multiple representation scales with learned fusion
50
- - **Cross-Contrastive Learning**: Aligns patch-level features with crystal anchors
51
-
52
- ## Architecture
53
 
54
  ```
55
- Image [B, 3, 32, 32]
56
- β”‚
57
- β–Ό
58
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
59
- β”‚ BEANS BACKBONE β”‚
60
- β”‚ β”œβ”€ Patch Embed β†’ [64 patches, 512d]
61
- β”‚ β”œβ”€ Hybrid Cantor Router (Ξ±=0.3)
62
- β”‚ β”œβ”€ 8 Γ— Attention Blocks (8 heads)
63
- β”‚ └─ 8 Γ— Pentachoron Expert Layers
64
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
65
- β”‚
66
- β–Ό
67
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
68
- β”‚ DAVID HEAD β”‚
69
- β”‚ β”œβ”€ Multi-scale projection: [256, 512, 768]
70
- β”‚ β”œβ”€ Per-scale Crystal Heads
71
- β”‚ └─ Geometric Fusion (learned weights)
72
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
73
- β”‚
74
- β–Ό
75
- [100 classes]
76
- ```
77
-
78
- ## Training Details
79
-
80
- | Parameter | Value |
81
- |-----------|-------|
82
- | Dataset | CIFAR-100 |
83
- | Classes | 100 |
84
- | Image Size | 32Γ—32 |
85
- | Patch Size | 4Γ—4 |
86
- | Embedding Dim | 512 |
87
- | Layers | 8 |
88
- | Attention Heads | 8 |
89
- | Experts | 5 (pentachoron) |
90
- | Sparse Neighbors | k=32 |
91
- | Scales | [256, 512, 768] |
92
- | Epochs | 200 |
93
- | Batch Size | 128 |
94
- | Learning Rate | 0.0005 |
95
- | Weight Decay | 0.1 |
96
- | Mixup Ξ± | 0.3 |
97
- | CutMix Ξ± | 1.0 |
98
- | Label Smoothing | 0.1 |
99
-
100
- ## Results
101
-
102
- | Metric | Value |
103
- |--------|-------|
104
- | **Top-1 Accuracy** | **70.05%** |
105
-
106
- ## TensorBoard Logs
107
-
108
- Training logs are included in the `tensorboard/` directory. To view:
109
-
110
- ```bash
111
- tensorboard --logdir tensorboard/
112
  ```
113
 
114
  ## Usage
115
 
116
  ```python
117
- import torch
118
  from safetensors.torch import load_file
119
  from david_beans import DavidBeans, DavidBeansConfig
 
 
 
 
120
 
121
  # Load config
122
- config = DavidBeansConfig(
123
- image_size=32,
124
- patch_size=4,
125
- dim=512,
126
- num_layers=8,
127
- num_heads=8,
128
- num_experts=5,
129
- k_neighbors=32,
130
- cantor_weight=0.3,
131
- scales=[256, 512, 768],
132
- num_classes=100
133
- )
134
-
135
- # Create model and load weights
136
  model = DavidBeans(config)
137
- state_dict = load_file("model.safetensors")
138
  model.load_state_dict(state_dict)
139
 
140
  # Inference
@@ -144,16 +62,37 @@ with torch.no_grad():
144
  predictions = output['logits'].argmax(dim=-1)
145
  ```
146
 
147
- ## Citation
 
 
 
 
 
 
 
148
 
149
- ```bibtex
150
- @misc{davidbeans2025,
151
- author = {AbstractPhil},
152
- title = {DavidBeans: Unified Vision-to-Crystal Architecture},
153
- year = {2025},
154
- publisher = {HuggingFace},
155
- url = {https://huggingface.co/AbstractPhil/geovit-david-beans}
156
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
157
  ```
158
 
159
  ## License
 
9
  - cantor-routing
10
  - pentachoron
11
  - multi-scale
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  ---
13
 
14
+ # πŸ«˜πŸ’Ž DavidBeans: Unified Vision-to-Crystal Architecture
15
 
16
+ This repository contains training runs for DavidBeans - a unified geometric deep learning architecture combining:
17
 
18
+ - **BEANS (ViT Backbone)**: Cantor-routed sparse attention
19
+ - **DAVID (Classifier)**: Multi-scale crystal projection with Cayley-Menger geometric regularization
20
 
21
+ ## Repository Structure
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
 
23
  ```
24
+ AbstractPhil/geovit-david-beans/
25
+ β”œβ”€β”€ README.md (this file)
26
+ └── weights/
27
+ β”œβ”€β”€ run_001_baseline_YYYYMMDD_HHMMSS/
28
+ β”‚ β”œβ”€β”€ best.safetensors
29
+ β”‚ β”œβ”€β”€ epoch_010.safetensors
30
+ β”‚ β”œβ”€β”€ config.json
31
+ β”‚ β”œβ”€β”€ training_config.json
32
+ β”‚ └── tensorboard/
33
+ β”œβ”€β”€ run_002_5expert_5scale_YYYYMMDD_HHMMSS/
34
+ β”‚ └── ...
35
+ └── ...
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
  ```
37
 
38
  ## Usage
39
 
40
  ```python
 
41
  from safetensors.torch import load_file
42
  from david_beans import DavidBeans, DavidBeansConfig
43
+ import json
44
+
45
+ # Pick a run
46
+ run_path = "weights/run_002_5expert_5scale_20251129_171229"
47
 
48
  # Load config
49
+ with open(f"{run_path}/config.json") as f:
50
+ config_dict = json.load(f)
51
+ config = DavidBeansConfig(**config_dict)
52
+
53
+ # Load model
 
 
 
 
 
 
 
 
 
54
  model = DavidBeans(config)
55
+ state_dict = load_file(f"{run_path}/best.safetensors")
56
  model.load_state_dict(state_dict)
57
 
58
  # Inference
 
62
  predictions = output['logits'].argmax(dim=-1)
63
  ```
64
 
65
+ ## Training Runs
66
+
67
+ | Run | Name | Accuracy | Notes |
68
+ |-----|------|----------|-------|
69
+ | 001 | baseline | 70.05% | Initial CIFAR-100 run |
70
+ | 002 | 5expert_5scale | 68.34% | 5 experts, 5 scales |
71
+
72
+ ## Architecture
73
 
74
+ ```
75
+ Image [B, 3, 32, 32]
76
+ β”‚
77
+ β–Ό
78
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
79
+ β”‚ BEANS BACKBONE β”‚
80
+ β”‚ β”œβ”€ Patch Embed β†’ [64 patches, dim] β”‚
81
+ β”‚ β”œβ”€ Hybrid Cantor Router β”‚
82
+ β”‚ β”œβ”€ N Γ— Attention Blocks β”‚
83
+ β”‚ └─ N Γ— Pentachoron Expert Layers β”‚
84
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
85
+ β”‚
86
+ β–Ό
87
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
88
+ β”‚ DAVID HEAD β”‚
89
+ β”‚ β”œβ”€ Multi-scale projection β”‚
90
+ β”‚ β”œβ”€ Per-scale Crystal Heads β”‚
91
+ β”‚ └─ Geometric Fusion β”‚
92
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
93
+ β”‚
94
+ β–Ό
95
+ [num_classes]
96
  ```
97
 
98
  ## License