ViperEkura
/
AstrAI
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs(data/dataset): 更新 load_mmap_files 函数的文档

This commit is contained in:
ViperEkura 2025-11-28 20:27:57 +08:00
parent 1f5cba889b
commit 567c55685e
1 changed files with 14 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
khaosz/data/dataset.py
View File

@ -19,6 +19,20 @@ def load_mmap_files(root_path: str, shared: bool=True) -> Tuple[MultiSeg, int]:
corresponding binary files as memory-mapped tensors. Returns tensors grouped by key corresponding binary files as memory-mapped tensors. Returns tensors grouped by key
and total number of elements. and total number of elements.
json metadata like this:
```
[
{
"file_name": "file1.bin",
"size": 1000,
"dtype": "float32",
"key": "key1"
},
...
]
```
Args: Args:
root_path: Root directory path containing file_mapper.json and binary files root_path: Root directory path containing file_mapper.json and binary files
shared: Whether to load tensors in shared mode. If True, tensors can be shared: Whether to load tensors in shared mode. If True, tensors can be