diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md new file mode 100644 index 0000000..f3fe9db --- /dev/null +++ b/.github/ISSUE_TEMPLATE.md @@ -0,0 +1,41 @@ +--- +name: Issue Report +about: Report a bug or suggest a feature +title: '[ISSUE] ' +labels: '' +assignees: '' +--- + +## Type +Please select the type of this issue: +- [ ] Bug Report +- [ ] Feature Request + +## Description +A clear and concise description of the issue or suggestion. + +## Steps to Reproduce (for bugs) +1. +2. +3. + +## Expected Behavior (for bugs) +What you expected to happen. + +## Actual Behavior (for bugs) +What actually happened. + +## Solution Proposal (for features) +Describe the solution you'd like. + +## Alternatives Considered (for features) +Describe any alternative solutions or features you've considered. + +## Environment (for bugs) +- Python version: +- AstrAI version (or commit hash): +- OS: +- GPU (if applicable): + +## Additional Context +Add any other context, screenshots, or logs here. \ No newline at end of file diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 0000000..9b1bbcf --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,26 @@ +## Description +Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. + +Fixes # (issue number) + +## Type of Change +Please delete options that are not relevant. + +- [ ] Bug fix (non-breaking change which fixes an issue) +- [ ] New feature (non-breaking change which adds functionality) +- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) +- [ ] Documentation update +- [ ] Other (please describe): + +## How Has This Been Tested? +Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. + +## Checklist: +- [ ] My code follows the style guidelines of this project (run `ruff format .` and `ruff check --fix .`) +- [ ] I have performed a self-review of my own code +- [ ] I have commented my code, particularly in hard-to-understand areas +- [ ] I have made corresponding changes to the documentation +- [ ] My changes generate no new warnings +- [ ] I have added tests that prove my fix is effective or that my feature works +- [ ] New and existing unit tests pass locally with my changes +- [ ] Any dependent changes have been merged and published in downstream modules \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..2030062 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,68 @@ +# Contributing to AstrAI + +Thank you for your interest in contributing to AstrAI! This document provides guidelines and steps for contributing. + +## How to Contribute + +### Reporting Issues +If you encounter a bug or have a feature request, please open an issue on GitHub. Include as much detail as possible: +- A clear description of the problem or request. +- Steps to reproduce (for bugs). +- Your environment (Python version, OS, etc.). + +### Submitting Changes +1. **Fork** the repository. +2. **Clone** your fork: + ```bash + git clone https://github.com/your-username/AstrAI.git + cd AstrAI + ``` +3. **Create a feature branch**: + ```bash + git checkout -b feature/your-feature-name + ``` +4. **Make your changes**. Follow the code style guidelines below. +5. **Commit your changes** with a descriptive commit message: + ```bash + git commit -m "Add: brief description of the change" + ``` +6. **Push** to your fork: + ```bash + git push origin feature/your-feature-name + ``` +7. **Open a Pull Request** (PR) against the `main` branch of the upstream repository. + +## Code Style + +AstrAI uses [Ruff](https://docs.astral.sh/ruff/) for code formatting and linting. Please ensure your code is formatted before submitting. + +- Run Ruff to format and lint: + ```bash + ruff format . + ruff check --fix . + ``` +- The project uses **double quotes** for strings and **4‑space indentation** (as configured in `pyproject.toml`). + +## Testing + +If you add or modify functionality, please include appropriate tests. + +- Run the test suite with: + ```bash + pytest + ``` +- Ensure all tests pass before submitting your PR. + +## Code Review + +All submissions will be reviewed. We may request changes or discuss alternatives. Please be responsive to feedback. + +## License + +By contributing, you agree that your contributions will be licensed under the same [GPL-3.0 License](LICENSE) that covers the project. + +--- + +If you have any questions, feel free to ask in the [GitHub Discussions](https://github.com/ViperEkura/AstrAI/discussions) or open an issue. + +Happy contributing! \ No newline at end of file diff --git a/README.md b/README.md index 818d5c0..798ae32 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@
English • - 中文 + 中文

@@ -13,23 +13,37 @@

-## 📖 Table of Contents | 目录 +
+ python + license + release + stars + forks +
+ +
+ +
+ English • + 中文 • + Issue Tracker • + Discussions • + HuggingFace +
+ +
+ +## 📖 Table of Contents
English -- [Installation](#installation) +- [Features](#features) - [Quick Start](#quick-start) - [Documentation](#documentation) - -
- -
-中文 - -- [安装](#安装) -- [快速开始](#快速开始) -- [文档](#文档) +- [Contributing](#contributing) +- [Community](#community) +- [License](#license)
@@ -40,59 +54,16 @@ ### Features -- 🚀 **High Performance**: Optimized for both training and inference -- 🔧 **Flexible**: Support for seq/sft/dpo training -- 💡 **Easy to Use**: Simple API with comprehensive examples -- 📦 **Lightweight**: Minimal dependencies - -### Installation - -```bash -git clone https://github.com/ViperEkura/AstrAI.git -cd AstrAI -pip install -e . -``` +- 🚀 **High Performance**: Optimized for both training and inference with efficient parallelization. +- 🔧 **Flexible**: Support for seq/sft/dpo training, customizable model architectures. +- 💡 **Easy to Use**: Simple API with comprehensive examples and demos. +- 📦 **Lightweight**: Minimal dependencies, easy to deploy. +- 🔬 **Research‑Friendly**: Modular design, easy to experiment with new ideas. +- 🤗 **HuggingFace Integration**: Compatible with HuggingFace models and datasets. ### Quick Start -```bash -# Train -python tools/train.py \ - --train_type=seq \ - --data_root_path=/path/to/dataset \ - --param_path=/path/to/param_path - -# Generate -python tools/generate.py --param_path=/path/to/param_path -``` - -### Demo - -```bash -# run download before using -python demo/download.py - -# run demo -python demo/stream_chat.py -python demo/generate_batch.py -python demo/generate_ar.py -``` - -- [bilibili](https://www.bilibili.com/video/BV1z5RPYHEkd) - ---- - - -## 中文 - -### 特性 - -- 🚀 **高性能**: 训练与推理双向优化 -- 🔧 **灵活**: 支持 seq/sft/dpo 多种训练方式 -- 💡 **易用**: 简洁的 API 与丰富的示例 -- 📦 **轻量**: 依赖少，部署简单 - -### 安装 +#### Installation ```bash git clone https://github.com/ViperEkura/AstrAI.git @@ -100,52 +71,79 @@ cd AstrAI pip install -e . ``` -### 快速开始 +For development dependencies: + +```bash +pip install -e ".[dev]" +``` + +#### Train a Model ```bash -# 训练 python tools/train.py \ --train_type=seq \ --data_root_path=/path/to/dataset \ --param_path=/path/to/param_path +``` -# 生成 +#### Generate Text + +```bash python tools/generate.py --param_path=/path/to/param_path ``` -### 演示 +#### Demo + +Check out the demos in the `demo/` folder: ```bash -# 使用前先下载模型 +# Download pre‑processed data (required before running demos) python demo/download.py -# 运行示例 +# Interactive streaming chat python demo/stream_chat.py + +# Batch generation python demo/generate_batch.py + +# Auto‑regressive generation python demo/generate_ar.py ``` -- [bilibili](https://www.bilibili.com/video/BV1z5RPYHEkd) +Watch a video walkthrough on [bilibili](https://www.bilibili.com/video/BV1z5RPYHEkd). - ---- - - - -### Document | 文档 +### Documentation | Document | Description | |----------|-------------| -| *Parameter Guide*
[参数说明](./assets/docs/params.md) | *Training & inference parameters*
训练与推理参数配置 | -| *Design Document*
[设计文档](./assets/docs/design.md) | *Framework architecture & module design*
系统架构与模块设计 | -| *Data Flow*
[数据流程](./assets/docs/dataflow.md) | *Data processing pipeline details*
数据处理管道详解 | -| *Model Introduction*
[模型介绍](./assets/docs/introduction.md) | *Model architecture & technical details*
模型架构与技术细节 | +| [Parameter Guide](./assets/docs/params.md) | Training & inference parameters | +| [Design Document](./assets/docs/design.md) | Framework architecture & module design | +| [Data Flow](./assets/docs/dataflow.md) | Data processing pipeline details | +| [Model Introduction](./assets/docs/introduction.md) | Model architecture & technical details | -### Download | 下载 +### Contributing -- [HuggingFace](https://huggingface.co/ViperEk/AstrAI) -- `python demo/download.py` +We welcome contributions! Please see our [Contributing Guidelines](CONTRIBUTING.md) for details. -### Lincence | 许可证 +1. Fork the repository. +2. Create a feature branch. +3. Commit your changes. +4. Open a Pull Request. -- [GPL-3.0](LICENSE) \ No newline at end of file +For major changes, please open an issue first to discuss what you would like to change. + +### Community + +- **GitHub Issues**: [Issue Tracker](https://github.com/ViperEkura/AstrAI/issues) +- **Discussions**: [GitHub Discussions](https://github.com/ViperEkura/AstrAI/discussions) +- **HuggingFace**: [Model Hub](https://huggingface.co/ViperEk/AstrAI) + +### License + +This project is licensed under the [GPL-3.0 License](LICENSE). + +--- + +
+ A lightweight Transformer framework designed for both high performance and ease of use. +
\ No newline at end of file diff --git a/assets/docs/README-zh-CN.md b/assets/docs/README-zh-CN.md new file mode 100644 index 0000000..b976748 --- /dev/null +++ b/assets/docs/README-zh-CN.md @@ -0,0 +1,144 @@ +
+ + +

AstrAI

+ +
+ English • + 中文 +
+ +

+ 轻量级 Transformer 训练与推理框架 +

+
+ +
+ python + license + release + stars + forks +
+ +
+ +
+ English • + 中文 • + 问题追踪 • + 讨论区 • + HuggingFace +
+ +
+ +## 📖 目录 + +- [特性](#特性) +- [快速开始](#快速开始) +- [文档](#文档) +- [贡献](#贡献) +- [社区](#社区) +- [许可证](#许可证) + +--- + + +## 中文 + +### 特性 + +- 🚀 **高性能**: 训练与推理双向优化，高效并行。 +- 🔧 **灵活**: 支持 seq/sft/dpo 多种训练方式，可定制模型架构。 +- 💡 **易用**: 简洁的 API 与丰富的示例、演示。 +- 📦 **轻量**: 依赖少，部署简单。 +- 🔬 **研究友好**: 模块化设计，便于实验新想法。 +- 🤗 **HuggingFace 集成**: 兼容 HuggingFace 模型与数据集。 + +### 快速开始 + +#### 安装 + +```bash +git clone https://github.com/ViperEkura/AstrAI.git +cd AstrAI +pip install -e . +``` + +安装开发依赖： + +```bash +pip install -e ".[dev]" +``` + +#### 训练模型 + +```bash +python tools/train.py \ + --train_type=seq \ + --data_root_path=/path/to/dataset \ + --param_path=/path/to/param_path +``` + +#### 文本生成 + +```bash +python tools/generate.py --param_path=/path/to/param_path +``` + +#### 演示 + +查看 `demo/` 文件夹中的演示： + +```bash +# 下载预处理数据（运行演示前必需） +python demo/download.py + +# 交互式流式聊天 +python demo/stream_chat.py + +# 批量生成 +python demo/generate_batch.py + +# 自回归生成 +python demo/generate_ar.py +``` + +观看 [bilibili](https://www.bilibili.com/video/BV1z5RPYHEkd) 上的视频演示。 + +### 文档 + +| 文档 | 说明 | +|------|------| +| [参数说明](./params.md) | 训练与推理参数配置 | +| [设计文档](./design.md) | 系统架构与模块设计 | +| [数据流程](./dataflow.md) | 数据处理管道详解 | +| [模型介绍](./introduction.md) | 模型架构与技术细节 | + +### 贡献 + +我们欢迎贡献！请参阅[贡献指南](../../CONTRIBUTING.md)了解详情。 + +1. Fork 本仓库。 +2. 创建功能分支。 +3. 提交更改。 +4. 发起 Pull Request。 + +重大更改请先开 issue 讨论。 + +### 社区 + +- **GitHub Issues**: [问题追踪](https://github.com/ViperEkura/AstrAI/issues) +- **Discussions**: [GitHub 讨论区](https://github.com/ViperEkura/AstrAI/discussions) +- **HuggingFace**: [模型中心](https://huggingface.co/ViperEk/AstrAI) + +### 许可证 + +本项目采用 [GPL-3.0 许可证](../../LICENSE)。 + +--- + +
+ 专为高性能与易用性设计的轻量级 Transformer 框架。 +
\ No newline at end of file