-
-
AstrAI
+
-
## 📖 目录
@@ -131,7 +129,7 @@ python scripts/demo/generate_ar.py
- **GitHub Issues**: [问题追踪](https://github.com/ViperEkura/AstrAI/issues)
- **Discussions**: [GitHub 讨论区](https://github.com/ViperEkura/AstrAI/discussions)
-- **HuggingFace**: [模型中心](https://huggingface.co/ViperEk/AstrAI)
+- **HuggingFace**: [模型中心](https://huggingface.co/ViperEk)
### 许可证
diff --git a/assets/docs/introduction.md b/assets/docs/introduction.md
index b7fc12f..9d5e4a6 100644
--- a/assets/docs/introduction.md
+++ b/assets/docs/introduction.md
@@ -1,12 +1,54 @@
## Model Introduction
-
-
### 1. Model Architecture
This model uses the Transformer architecture with GQA mechanism (q_head=24, kv_head=4), which saves KV cache memory compared to traditional MHA (although KV cache is not currently implemented). The model is built by stacking 24 layers of Transformer blocks, with 1.0 billion parameters. Transformer is an autoregressive model that calculates the relationship between all previous tokens to obtain the probability distribution of the next token.
-
+```mermaid
+flowchart TB
+ subgraph Layers["Transformer Layers"]
+ direction TB
+ A[Input Embedding] --> B[Transformer Block\nLayer 1]
+ B --> C[Transformer Block\nLayer ...]
+ C --> D[Transformer Block\nLayer 32]
+ D --> E[RMSNorm]
+ E --> F[Linear]
+ F --> G[SoftMax]
+ end
+
+ subgraph TransformerBlock["Transformer Block"]
+ direction TB
+ H[x] --> I[RMSNorm]
+ I --> J[Linear → Q/K/V]
+ J --> K[Q]
+ J --> L[K]
+ J --> M[V]
+ K --> N[RoPE]
+ L --> O[RoPE]
+ N --> P["Q @ K^T / sqrt(d)"]
+ O --> P
+ P --> Q[Masked SoftMax]
+ Q --> R[S @ V]
+ M --> R
+ R --> S[Linear]
+ S --> T[+]
+ H --> T
+ T --> U[RMSNorm]
+ U --> V[Linear]
+ V --> W[SiLU]
+ V --> X[×]
+ W --> X
+ X --> Y[Linear]
+ Y --> Z[+]
+ T --> Z
+ Z --> AA[x']
+ end
+
+ classDef main fill:#e6f3ff,stroke:#0066cc;
+ classDef block fill:#fff2e6,stroke:#cc6600;
+ class Layers main;
+ class TransformerBlock block;
+```
What is an autoregressive model? After splitting a sentence into tokens, the model predicts the probability distribution of the next token. This means the model calculates the probability of the next possible token and its corresponding probability based on the given context (the sequence of tokens that have already appeared).
diff --git a/assets/images/logo.png b/assets/images/logo.png
new file mode 100644
index 0000000..a8987fb
Binary files /dev/null and b/assets/images/logo.png differ
diff --git a/assets/images/project_logo.png b/assets/images/project_logo.png
deleted file mode 100644
index 25cecdb..0000000
Binary files a/assets/images/project_logo.png and /dev/null differ
diff --git a/assets/images/structure.png b/assets/images/structure.png
deleted file mode 100644
index 5f4f3a2..0000000
Binary files a/assets/images/structure.png and /dev/null differ
